1. KANAL: Knowledge ANALysis
Jihie Kim
Yolanda Gil
USC/ISI

2. Role of Knowledge Analysis in SRI Team
To point out to the Interaction Manager what additional K needs to be acquired or what existing K needs to be modified
To guard the knowledge server from invalid statements entered by the user

3. Approach: Using Interdependency Models
Relating different pieces of Knowledge among themselves and to the existing KB
(e.g., how different pieces of knowledge are put together to generate an answer)
Successfully used in checking problem-solving K in EXPECT (Gil & Melz 96; Kim & Gil 99)

4. Current Focus: Checking Process Models
Verification checks: model is correct (e.g., no steps missing
Validation checks: model is as user intended (e.g., alert user of impossible paths)
KANAL
Interaction
Manager UI KM
Interaction Plans
for fixing errors

5. Validating Complex Process Models
Lambda Virus Invasion 2 … …
Transcribe
Assemble
Enter …
Replicate
Arrive …
Integrate
Divide
Disintegrate
Circularize
Synthesize
Copy

6. Describing Process Models (Composed Concepts)
Each individual step has
Preconditions, Add-list, Delete-list
Links among the steps
Decomposition links between steps and substeps
Disjunctive alternatives
Temporal links …
VirusInvasion
substeps
Enter
Integrate
Synthesize
. . .
disjunction

7. Checks on Process Models
All the steps are properly linked (substep, nextstep, disjunctive nextstep, conjunctive nextstep, …)
All the preconditions of each step are satisfied during the simulation
All the expected effects can be achieved
There are no unexpected effects
There are no impossible paths
. . .

8. Current Focus: Dynamic Checks
Simulation (or symbolic execution) results show how substeps of the process model are related each other (Interdependency Model)
Perform various kinds of checks
unachieved preconditions
expected/unexpected effects
disjunctive branches
loops
causal links
redundancies
unordered steps …
: Implemented

9. Checking Unachieved Preconditions
During simulation, collect unachieved preconditions by tracing failed expressions
Suggest fixes
Add a step that can achieve the condition
Add ordering constraints between the failed step and another step that undid the condition
Delete the step
. . .
VirusInvasion
Failed Precondition:
Virus near Cell
Proposed Fixes:
Add an Arrive step
Add a Move step
. . .
Enter
Integrate

10. Checking Effects Compute the effects by simulation
Suggest fixes for unachieved expected effects
Add steps that can achieve the effect
Add ordering constraints between effect adding steps and effect deleting steps
Check unexpected effects
After VirusInvasion
ProteinCoat of the virus broken  Achieved
DNA of the virus has replicates  Unachieved
<Proposed Fixes>
Add a Replicate step
Add a Divide step

11. Checking Disjunctive Branches
Inform all the combinations of alternatives so that the user can check if some are impossible
KANAL can simulate and highlight disjunctive paths

12. Example: Lambda Virus Invasion
(From Alberts ECB Chapter 9)
Enter
Circularize
Integrate
Divide
Disintegrate
Synthesize
Replicate
disjunction
Arrive
<Paths Simulated>
Path1: Arrive1  Enter2  Circularize3  Integrate4  Divide5
 Disintegrate6  Synthesize7  Replicate8
Path2: Arrive1  Enter2  Circularize3  Synthesize7  Replicate8

13. Example: Conjunctive Branches
Life cycle of a virus (from Alberts ECB Chapter 9)
Transcribe
Arrive
Enter
Conjunction
Assemble
Replicate
<Simulation sequence>
Arrive1  Enter2  Trascribe3  Replicate4  Assembly5
Arrive1  Enter2  Replicate4  Trascribe3  Assembly5

14. Checking Loops Enter Circularize Integrate Divide Disintegrate
Synthesize
Replicate
disjunction
Arrive
<Loops Found>
Loop1: Arrive1  Enter2  Circularize3  Integrate4  Divide5
 Disintegrate6  Synthesize7  Replicate8 Arrive1
Loop2: Arrive1 Enter2  Circularize3  Synthesize7  Replicate8
 Enter1
Loop3: Divide5 Divide5

15. Checking Causal Links Describe which step enables (or disables)
a given step
Enter
Circularize
Integrate
Divide
Disintegrate
Synthesize
Replicate
disjunction
Arrive
<Causal Links>
Arrive1 enables Enter2 by achieving “Virus near Cell”
Integrate4 enables Disintegrate6 by achieving
“Virus DNA integrated with chromosome”

16. Fixing Problems: Using Interaction Plans
Interaction Plan: describes how to proceed with the user interaction
direct what to do next based on the results from K Analysis
KANAL’s dialogue for fixing errors is implemented with interaction plans
Will be integrated with the Interaction Manager

17. Keeping Track of Interaction History
...
Choose what to simulate
choose model: VirusInvadesCell
choose substep to test: VirusInvadesCell
Simulate model VirusInvadesCell
simulate-steps-&-find-failed-events
ask-to-fix-failed-event: (failed preconditions of Enter)
propose-fixes-for-failed-event
ask-what-to-fix-for-failed-event :
((the location of (the patient of Enter)) =
(the space-near of (the agent of Enter)))
ask-how-to-fix-failed-event (add Arrive before Enter)

18. Future Extensions (I): Static Checks
Let user pose questions about various features of the process model to test the model
KANAL will maintain test suites
Users pick from sample query templates
example: retrieving role values, part-of relations, type definitions,..
Users may specify their expected results
Users may vary the initial situations to start from
Explanation or trace of the answer to a query show how different pieces of K are used to generate the answer (Interdependency Model)

19. Future Extensions (II)
Exploiting history and evolution of Interdependency Models (for both simulations and queries)
Example: Check what tests were correctly answered before
Using heuristics to focus K analysis
Example: when invalid results are obtained, KANAL will use a divide-and-conquer strategy and check intermediate results to find the sources of the problem
Testing with different initial states and different arguments

20. Future Extensions (III)
Interdependency Models for problem solving knowledge
EKCP
Build on past work on EXPECT

21. Using KANAL for Intelligent Tutoring Systems
ITSs can acquire domain knowledge from human instructor and use simulations to refine the knowledge (Johnson et al 2000, Scholer et al 2000, Angros et al 99)
We are exploring the use of KANAL to check and analyze the domain models while it is being built

22. Knowledge Authoring Environment for Tutoring Systems (current)
Instructor
Student
Demonstration
Final
Model
(Lessons)
Steve
Agent
Initial
Model
Refined
Model
Experimenter
Library
of actions
Domain
Simulator

23. Knowledge Authoring Environment for Tutoring Systems (future)
Instructor
Student
Demonstration
Editor
Final
Model
(Lessons)
Steve
Agent
Initial
Model
Refined
Model
Experimenter
KANAL
Library
of actions
Domain
Simulator