1. Made with Protégé: An Intelligent Medical Training System Olga Medvedeva, Eugene Tseytlin, and Rebecca Crowley Center for Pathology Informatics, University of Pittsburgh School of Medicine, Pittsburgh PA The SlideTutor Project Pedagogic Knowledge Representation References Acknowledgements http://slidetutor.upmc.edu “I hear and I forget, I see and I remember, I do and I understand” – Chinese Proverb Learning by doing is an extremely effective instructional method, but training novices in practice environments is not an option in high-risk domains such as Air Traffic Control and Medicine. Intelligent Tutoring Systems provide environments for learning by doing, which have proven to be highly effective, but have not yet been applied in domains that are knowledge-rich. Q: Can Knowledge-Based Systems (KBS) be used as underlying component for developing such a training environment? The SlideTutor project is developing a KBS based Intelligent Training System for learning microscopic diagnosis. We are simultaneously examining this question from four perspectives: technical, cognitive, design, and educational. This demonstration and poster highlights some of our progress in each of these dimensions. Cognitive & developmental model The design of SlideTutor is based on a developmental cognitive model of visual classification problem solving derived from our own empirical studies of classification problem solving in microscopic diagnosis, as well as numerous other studies of expertise in visual and non-visual domains. Derived Principles Architecture Based on the implications of our cognitive and developmental model, we designed our system, to: (1) determine both correct and incorrect student actions, determine the general class of error that has been made, and leave open the specific instructional response to this error. (2) reason with the student, in order to provide correct feedback as intermediate solutions are developed. For example, the system should accept hypotheses and even diagnoses based on an early, incomplete or incompletely refined set of evidence. When additional evidence is identified, the system should require that students revise their diagnoses or statements of diagnostic support. (3) reason both forwards (data to hypothesis) and backwards (hypothesis to data) so that it can support both strategies among students. Three important considerations led to our need for a highly modular architecture: (1) deep structural similarities between all visual classification tasks, (2) need for a completely separate and therefore flexible pedagogic model, and (3) requirement for parametric testing of learning gains. We chose to use the UPML Component model as the basis for our architecture. Domain Knowledge Representation On the expert model side, we used Protégé to implement and slightly extend Motta et al’s representation of classification problem solving. Projects for Case and Domain include a thirds project which contains features, attributes, and values common to both the case and knowledge representation. Classes and instances from these KB populate the Jess-based Dynamic Solution Graph. Dynamic Solution Graph To create our virtual slide case projects, we created a Protégé plugin that utilizes the same Domain KB as the Tutor. When a new slide is authored, the author indicates the diagnosis and FEATURE_SPECIFICATION from the KB that represents the current case. The authoring system automatically fills instances of ATTRIBUTE_VALUE_PAIR and OBSERVABLE. When authors draw around particular visual features they choose the correct OBSERVABLE from the list of those associated with the FEATURE_SPECIFICATION, which is saved in combination with the location as a LOCATED_OBSERVABLE. We have created an initial set of 30 cases in Subepidermal Vesicular Dermatitides. A Protégé Plugin for Authoring The Dynamic Solution Graph (DSG) is a directed acyclic graph that models the current problem state and all valid-next-steps. It exists only in Jess Working Memory. The DSG generates the initial problem state at the beginning of each problem, using knowledge derived from the Domain, Task and Case models. After each student action, the graph structure is updated by a set of abstract problem solving methods that may add or delete nodes and arcs, or change the node state. Any individual state of the DSG represents only the current problem state and all valid next-steps. But taken together, sequential DSG states define one path through the problem space – the path taken by the student. The graph representation supports the ability to reason in both directions (design principle 3) and the dynamic nature of the graph enables the system to reason with the student (design principle 2). Instructional Layer Tutor Interfaces Like our domain knowledge, pedagogic knowledge is modeled in Protégé and then loaded into Jess, using a modified version of JessTab. Examples of bug scenarios for Hypothesis Evaluation are shown below: The instructional layer is composed of Jess Rules instantiated with pedagogic knowledge, which implement the strategies derived from our cognitive and developmental model. We have developed two student interfaces that can be used by students and we are currently testing their effect on learning gains. The case-focused (CF) tutor uses simple graphical relationships between features and hypotheses. The knowledge-focused (KF) tutor presents an interactive algorithmic interface which is designed to co-construct knowledge with students. We are still in the midst of our first formative evaluation of the CF and KF interfaces. Early results from the CF tutor are very promising, demonstrating that students gain and retain on test which assess individual skills, and general diagnostic skills. We gratefully acknowledge contributions made to this project by: Elizabeth Legowski, Girish Chavan, Katsura Fujits, Maria Bond; students in the Interactive Design Course Winter, 2004; and the developers of Protégé, Jess, JessTab, SpaceTree, and JGraph. Learning gains