Presentation on theme: "Advanced Knowledge Technologies"— Presentation transcript:
1 Advanced Knowledge Technologies University of AberdeenUniversity of EdinburghUniversity of SheffieldOpen UniversityUniversity of SouthamptonMIAKTOxford UniversityKing’s College, LondonUniversity of SheffieldOpen UniversityUniversity of SouthamptonCoAKTinGUniversity of EdinburghOpen UniversityUniversity of Southampton
2 MIAKT: Medical Informatics and Knowledge Technologies Supporting triple-assessment (TA)(collaborative decision-making) for thediagnosis and treatmentof breast cancerOxford UniversityKings CollegeOpen UniversityUniversity of SheffieldUniversity of Southampton
3 To support collaboration for the e-Scientist Intelligent meeting spaces: Decision rationale, group memory capture Planning, coordination support Instant messaging/presenceOpen UniversityUniversity of EdinburghUniversity of Southampton
4 Advanced Knowledge Technologies Representation and ReasoningOntologies: domain and process modelsInteroperabilityIntegration with databases for scalability(Semantic Web)Reasoning services – local and distributedNatural language processingApplication domain – academic CS
5 People involved MIAKT Oxford University CoAKTinG Mike Brady, Jon WhitelyKing’s College LondonDavid Hawkes,Christine Tanner,Yalin ZhengThe Open UniversityEnrico Motta,John Domingue,Liliana CabralUniversity of SheffieldYorick Wilks,Fabio Ciravegna,Kalina BontchevaUniversity of SouthamptonNigel Shadbolt,Srinandan Dasmahapatra,Paul Lewis,Bo Hu,Hugh LewisCoAKTinGUniversity of EdinburghAustin TateStephen PotterJessica Chen-burgerJeff DaltonOpen UniversityMarc EisenstadSimon Buckingham ShumJiri KomzakMichelle BachlerUniversity of SouthamptonDavid De RoureNigel ShadboltDanius MichaelidesRichard BealesKevin PageBen Juby
6 Breast Cancer – Statistics & Screening EU: 24% of cancer cases19% of cancer deaths1 in 8 of women will develop breast cancer during the course of their lives1 in 28 will die of the disease.5 year survival rate for localized breast cancer is 97% for early detectionis 77% if the cancer has spread at diagnosisis 22% if distant metastases are foundScreening for ages 50+M Brady: MIAS (features), e-Diamond (priors)Knowledge technology support
8 Imaging: Mammography/Ultrasound/MR Mammography (X-ray)Position breast on small flat plate, with X-ray plate under it.Flat plate above your breast.When machine is switched on, breast pressed down between plates by machine to get clearest picture.Two pictures are taken: from above and from the side.Ultrasoundusually used for women under 35 (breasts too dense or solid to give a clear picture with mammography) It is also used to see if a breast lump is solid contains fluid (a cyst)
9 Histopathology Fine needle aspiration cytology Core biopsy With imaging guidance
11 Triple AssessmentClinical and radiological opinions are used independently to decide upon further interventionThe most suspicious opinion prevails (normal/definitely benign, probably benign, indeterminate, probably malignant)Needle biopsy is mandatory for all abnormalities classified as indeterminate or more suspiciousNeedle biopsy results are discussed in the context of imaging and clinical findings at multidisciplinary meetings
12 Radiology and Histopathology images: Different scales SamePatient:multipledescriptors
13 Ontology, Annotation, Language Generation Annotated images for retrieval in contextMemory aids for specialistsReport generation from annotationsOntologies for descriptive groundingCorrelative reasoning across specialisation(across scales orders of magnitude apart)Distributed reasoning (grid?)
14 Knowledge Engineering Lexical and ontological issuesDecision supportRecords – image and text
15 Diagnostic Mammography: different views To pinpoint exact size & location of breast abnormalityand to image surrounding tissue and lymph nodes.mediolaterallateromedialCranio-caudal (CC) & Mediolateral oblique (MLO) views
16 Diagnostic Mammography: BI-RADS Ontology - Masses 1. SHAPE1.a. Round1.b. Oval1.c. Lobular1.d. Irregular2. MARGINS2.a. Circumscribed2.b. Microlobulated2.c. Obscured2.d. Indistinct2.e. Spiculated3. DENSITY:3.a. High density3.b. Equal density3.c. Low density3.d. Fat containing – radiolucent –oil cyst, lipoma, or galactoceleas well as mixed lesions such ashamartoma or fibroadenolipoma.[and/or histologic terms]MASS: space occupying lesion seen in two different projections.If potential mass seen in single projection, called DENSITY until 3-D confirmation.
17 BI-RADS Ontology – Masses (details) MASS: space occupying lesion seen in two different projections.If potential mass seen in single projection, called DENSITY until 3-D confirmation.1. SHAPEa. Round: spherical, ball-shaped, circular or globularb. Oval: elliptical or egg-shaped.c. Lobular: has contours with undulations.d. Irregular: none of the above. 2. MARGINS [modify the shape of the mass]a. Circumscribed Margins: abrupt transition between the lesion and the surrounding tissue. Without additional modifiers there is nothing to suggest infiltration.b. Microlobulated Margins: undulate with short cycles producing small undulations.c. Obscured Margins: hidden by superimposed or adjacent normal tissue; cannot be assessed any further.d. Indistinct Margins: poor definition of margins raises concern of infiltration by the lesion; not likely due to superimposed normal breast tissue.e. Spiculated Margins: lines radiating from margins of mass3. DENSITY: x-ray attenuation of lesion relative to the expected attenuation of an equal volume of fibroglandular breast tissue; most cancers are of equal or higher density; never fat containing but may trap fat.a. High densityb. Equal densityc. Low densityd. Fat containing – radiolucent - oil cyst, lipoma, or galactocele as well as mixed lesions such as hamartoma or fibroadenolipoma. [When appropriate, histologic terms may be included]
18 MicrocalcificationMicrocalcifications: (<.5mm) specks of calcium in milk ducts.About half of the cancers detected by mammography appear as a cluster of microcalcifications.Microcalcifications are the most common mammographic sign of ductal carcinoma in situP(micro-Ca | DCIS)=0.9
19 BI-RADS: Calcifications Benign calcifications usually larger than malignant ones -- coarser, often round with smooth margins, more visible.Malignant calcifications very small, require magnifying glass.When specific aetiology not possible, description of calcifications should include their distribution and morphology.Benign calcifications only reported if judged to be susceptible to misinterpretation.
20 Multiple views on domain Structure, anatomyFunction, physiologyPathologyPatient history
21 BI-RADS Ontology: Calcification Typically benignIntermediate concernHigher probabilityDistribution modifiers
22 BI-RADS Ontology: Calcification 2. INTERMEDIATE CONCERN2.a. Amorphous or Indistinct Calcifications1. TYPICALLY BENIGN1.a. Skin Calcifications1.b. Vascular Calcifications1.c. Coarse Calcifications1.d. Large Rod-Like Calcifications1.e. Round Calcifications1.f. Lucent-Centered Calcifications1.g. Eggshell or Rim Calcifications1.h. Milk of Calcium Calcifications1.i. Suture Calcifications1.j. Dystrophic Calcifications1.k. Punctate Calcifications4. DISTRIBUTION MODIFIERS4.a. Grouped or Clustered4.b. Linear4.c. Segmental4.d. Regional:4.e.Diffuse/Scattered3. HIGHER PROBABILITY OF MALIGNANCY3.a. Pleomorphic or Heterogeneous Calcifications3.b. Fine, Linear or Branching Calcifications
23 BI-RADS Ontology: Calcification (details) TYPES AND DISTRIBUTION OF CALCIFICATION:1. TYPICALLY BENIGN -a. Skin Calcifications: typical lucent centered deposits that are pathognomonic. Atypical forms confirmed by tangential views to be in the skin.b. Vascular Calcifications: Parallel tracks, or linear tubular calcifications clearly associated with blood vessels.c. Coarse Calcifications: Classic calcifications produced by an involuting fibroadenoma.d. Large Rod-Like Calcifications: Continuous rods, occasionally branching, diameter > 1mm usually, may have lucent centers, if calcium surrounds rather than fills an ectactic duct. Found in secretory disease, "plasma cell mastitis", and duct ectasia.e. Round Calcifications: When multiple, of variable size. Considered benign and when small [under 1 mm], frequently formed in acini of lobules. Under 0.5 mm are termed punctate.f. Lucent-Centered Calcifications: Less that 1 mm to greater than 10 mm, smooth surfaces, round or oval, have lucent center. “Wall" thicker than the "rim or eggshell" type. Included are areas of fat necrosis, calcified debris in ducts, and occasional fibroadenomas.g. Eggshell or Rim Calcifications: Very thin, under 1mm thickness, appear as calcium deposited on the surface of a sphere. Although fat necrosis can produce these thin deposits, calcifications in the wall of cysts are the most common "rim" calcifications.h. Milk of Calcium Calcifications: Consistent with sedimented calcifications in cysts. Often less evident in craniocaudal image -- appear as fuzzy, round, amorphous deposits; sharply defined on 90° lateral -- semilunar, crescent shaped, curvilinear (concave up), or linear defining dependent portion of cysts.i. Suture Calcifications: Ca deposited on suture material, relatively common in post-irradiated breast, typically linear or tubular in appearance and knots are frequently visible.j. Dystrophic Calcifications: Usually form in irradiated breast or following trauma. Irregular in shape, usually > 0.5 mm, often have lucent centers.k. Punctate Calcifications: Round/oval, < 0.5 mm with well-defined margins.
24 Example Mammogram2 cm mass (tumour)microcalcifications
26 Histopathology Fine needle aspiration cytology Core biopsy With imaging guidanceFFNACDescriptors when drawing sample:mmSampling pattern for stereotactic FNAC
27 Histopathology slides A histological slide has an immense amount of data, the closer you look, the more there isHistological images are complex with challenges at both the segmentation and feature classification levelThink in terms of two scales – low-power, high-power
32 MIAKT: Technology Palette MIAS –Medical image registration (X-ray, MR)Segmentation and feature extractionImage ClassificationAKT –Ontology development(Distributed) reasoning servicesImage annotation against ontologiesNatural language generationDecision support – belief nets?
33 MIAKT Abstract away from the details of TA meeting Collaborative problem solving/decision makingPossibly distributed, virtual presenceWell-defined goals, well-defined contributory skill setsStructured protocolRequire recall of contents of events of past meetingReport generation (audit trail)
34 Enhance TechnologiesOntologically annotated audio/video streamsIssue handling, tasking, planning and coordinationCollective sense-making and group memory captureEnhanced presence management and visualisationAdaptive information systems
35 Technology Integration Aim: To support e-Science collaboration by integrating and demonstrating the utility of:intelligent task-orientated messaging, collaborative planning, issue, activity and constraint management (I-X Process Panels/<I-N-C-A>: Edinburgh)peripheral awareness of the online presence, availability, attributes and location of colleagues, documents, and devices (BuddySpace: OU)real time conversational mapping of meetings, providing shared visual focus and group memory capture (Compendium: OU)multimedia meeting mark-up, replay and navigation (HyStream: Southampton)
36 JabberJabber is a set of XML-based protocols for real-time messaging and presence notificationCommunicates with other instant messaging services through gatewaysMany clients available – see
38 CompendiumProvides a methodological framework, plus an evolving suite of tools, for collective sense-making and group memory.Intersection of collaborative modelling, organisational memory, computer-supported argumentation and meeting facilitation. Centres on face-to-face meetings, potentially the most pervasive knowledge-based activity in working life, but also one of the hardest to do well.
40 BuddySpace‘Enhanced Presence Management for Collaborative Working, Messaging, Gaming and Beyond’The concept of presence is a rich combination of attributes that characterise an individual's…physical and/or spatial locationwork trajectorytime frame of referencemental moodgoals and intentions
42 Process PanelsBased on notion of the representation of a product as a set of nodes making up the components of the product model, along with constraints on the relationship between those nodes and a set of outstanding issuesInvestigates the use of shared models for task directed communication between human and computer agents who are jointly exploring a range of alternative options for activity.
45 Smart spacesDevices in the room enables us to capture continuous (real time, multi-way, multi-cast, ontologically informed, …) metadataOther devices provide ‘presence’ informationConsider an experimental laboratory instead of a meeting room:InstrumentsElectronic log booksVisualisation