Multimedia Analytics Jianping Fan Department of Computer Science University of North Carolina at Charlotte.

Slides:

Advertisements

Similar presentations

Some Reflections on Augmented Cognition Eric Horvitz ISAT & Microsoft Research November 2000 Some Reflections on Augmented Cognition Eric Horvitz ISAT.

Advertisements

National Technical University of Athens Department of Electrical and Computer Engineering Image, Video and Multimedia Systems Laboratory

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki

Does one size really fit all? Evaluating classifiers in Bag-of-Visual-Words classification Christian Hentschel, Harald Sack Hasso Plattner Institute.

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Machine learning continued Image source:

Prof. Carolina Ruiz Computer Science Department Bioinformatics and Computational Biology Program WPI WELCOME TO BCB4003/CS4803 BCB503/CS583 BIOLOGICAL.

Visualization and Cluster

IJCAI Wei Zhang, 1 Xiangyang Xue, 2 Jianping Fan, 1 Xiaojing Huang, 1 Bin Wu, 1 Mingjie Liu 1 Fudan University, China; 2 UNCC, USA {weizh,

User and Task Analysis Howell Istance Department of Computer Science De Montfort University.

ACM Multimedia th Annual Conference, October , 2004

1 Integrating User Feedback Log into Relevance Feedback by Coupled SVM for Content-Based Image Retrieval 9-April, 2005 Steven C. H. Hoi *, Michael R. Lyu.

A novel log-based relevance feedback technique in content- based image retrieval Reporter: Francis 2005/6/2.

Active Learning with Support Vector Machines

OF INFORMATION AND KNOWLEDGE MITE INTERACTIVE REPRESENTATIONS 6323.

Announcements  Project teams should be decided today! Otherwise, you will work alone.  If you have any question or uncertainty about the project, talk.

T.Sharon-A.Frank 1 Multimedia Introduction and Motivation.

Learning to Match Ontologies on the Semantic Web AnHai Doan Jayant Madhavan Robin Dhamankar Pedro Domingos Alon Halevy.

Grand Challenges Robert Moorhead Mississippi State University Mississippi State, MS 39762

Presentation in IJCNN 2004 Biased Support Vector Machine for Relevance Feedback in Image Retrieval Hoi, Chu-Hong Steven Department of Computer Science.

Dept. of Computer Science & Engineering, CUHK Pseudo Relevance Feedback with Biased Support Vector Machine in Multimedia Retrieval Steven C.H. Hoi 14-Oct,

Presented by Zeehasham Rasheed

The Implicit Mapping into Feature Space. In order to learn non-linear relations with a linear machine, we need to select a set of non- linear features.

Course Instructor: Aisha Azeem

Jinhui Tang †, Shuicheng Yan †, Richang Hong †, Guo-Jun Qi ‡, Tat-Seng Chua † † National University of Singapore ‡ University of Illinois at Urbana-Champaign.

1. Human – the end-user of a program – the others in the organization Computer – the machine the program runs on – often split between clients & servers.

Teaching with Multimedia and Hypermedia

CS 8751 ML & KDDSupport Vector Machines1 Support Vector Machines (SVMs) Learning mechanism based on linear programming Chooses a separating plane based.

Chapter 1 Introduction to Data Mining

Taxonomies of Visualization Techniques CMPT 455/826 - Week 12, Day 2 w12d2 Sept-Dec

Personalized Interaction with Web Resources First Sino-German Symposium on KNOWLEDGE HANDLING: REPRESENTATION, MANAGEMENT AND PERSONALIZED APPLICATION.

HCI in Software Process Material from Authors of Human Computer Interaction Alan Dix, et al.

David S. Ebert David S. Ebert Visual Analytics to Enable Discovery and Decision Making: Potential, Challenges, and.

Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.

Glasgow 02/02/04 NN k networks for content-based image retrieval Daniel Heesch.

Markup and Validation Agents in Vijjana – A Pragmatic model for Self- Organizing, Collaborative, Domain- Centric Knowledge Networks S. Devalapalli, R.

Beauty is Here! Evaluating Aesthetics in Videos Using Multimodal Features and Free Training Data Yanran Wang, Qi Dai, Rui Feng, Yu-Gang Jiang School of.

The ISTE National Educational Technology Standards (NETS  S) and Performance Indicators for Students.

IBM QBIC: Query by Image and Video Content Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC 28223

Chapter 1 Foundations of Information Systems in Business.

Visual Analytics with Linked Open Data and Social Media for e- Governance Vitaveska Lanfranchi Suvodeep Mazumdar Tomi Kauppinen Anna Lisa Gentile Updated.

PhD CONCEPT Exploring Digital Maps as Communication Tools for eParticipation - On an example of Urban Planning PhD concept, Beate Weninger,

Face Detection Using Large Margin Classifiers Ming-Hsuan Yang Dan Roth Narendra Ahuja Presented by Kiang “Sean” Zhou Beckman Institute University of Illinois.

Lecture 4 TTH 03:30AM-04:45PM Dr. Jianjun Hu CSCE569 Parallel Computing University of South Carolina Department of.

Competence Centre on Information Extraction and Image Understanding for Earth Observation 29th March 2007 Category - based Semantic Search Engine 1 Mihai.

Digital Learning India 2008 July , 2008 Mrs. C. Vijayalakshmi Department of Computer science and Engineering Indian Institute of Technology – IIT.

Image Classification for Automatic Annotation

Image Classification over Visual Tree Jianping Fan Dept of Computer Science UNC-Charlotte, NC

Robots Unit Portfolio Presentation Ms Sabah Faisal.

CZ5225: Modeling and Simulation in Biology Lecture 7, Microarray Class Classification by Machine learning Methods Prof. Chen Yu Zong Tel:

L&I SCI 110: Information science and information theory Instructor: Xiangming(Simon) Mu Sept. 9, 2004.

Anomaly Detection in GPS Data Based on Visual Analytics Kyung Min Su - Zicheng Liao, Yizhou Yu, and Baoquan Chen, Anomaly Detection in GPS Data Based on.

The “Spatial Turing Test” Stephan Winter, Yunhui Wu

Supervised Machine Learning: Classification Techniques Chaleece Sandberg Chris Bradley Kyle Walsh.

Supporting team communication and coordination in visual analytics Narges Mahyar University of Victoria April 2014.

Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC Relevance Feedback for Image Retrieval.

Immersive Virtual Characters for Educating Medical Communication Skills J. Hernendez, A. Stevens, D. S. Lind Department of Surgery (College of Medicine)

Coached Active Learning for Interactive Video Search Xiao-Yong Wei, Zhen-Qun Yang Machine Intelligence Laboratory College of Computer Science Sichuan University,

Big Data Analytics and HPC Platforms

Inquiry learning and SimQuest

Compact Bilinear Pooling

Semantic Video Classification

Development of Assessment Literacy Knowledge Base

Personalized Social Image Recommendation

Metric Learning for Clustering

Project Implementation for ITCS4122

إستراتيجيات ونماذج التقويم

Ying Dai Faculty of software and information science,

Ying Dai Faculty of software and information science,

Ying Dai Faculty of software and information science,

Presentation transcript:

Multimedia Analytics Jianping Fan Department of Computer Science University of North Carolina at Charlotte

Presentation Outlines What is Multimedia Analytics in my mind? What multimedia analytics can do for Multimedia Computing? What we (UNCC team) have done so far?

What is Multimedia Analytics? Multimedia analytics is science of multimedia computing facilitated by visual interface for interactive user’s inputs & assessments! 1. Multimedia Rept. 2. Computing Hypos 3. Decision Function 4. Computing Results Visual Analytics Semantic Gap Machine Side Human Side Assess & Inputs multimedia computing 1. Multimedia Rept. 2. Computing Hypos 3. Decision Function 4. Computing Results

What is Multimedia Analytics? Multimedia Analytics a. Machine-Based Multimedia Computing; b. Visualization of Multimedia Data, Knowledge, Hypotheses, Decision Functions & Results; c. Human-Computer Interaction for assessing & changing hypotheses for multimedia computing Loop

What Multimedia Analytics can do for Multimedia Computing? Multimedia Analytics Multimedia Computing Knowledge & Hypotheses Visualization User-System Interaction & Assessment Decision making Decision adjustment Hypothesis visualization Hypothesis assessment sampling & projection Hypothesis updating

What Multimedia Analytics can do for Multimedia Computing? Hypothesis & Decision Function Visualization and Assessment Computing Results & Knowledge Visualization, Exploration and Assessment Leveraging the advantages of both human beings on creative thinking and computers on large memory and computing capacity.

What we have done so far a. Interactive Similarity Function Assessment

What we have done so far a. Interactive Similarity Function Assessment

What we have done so far a. Interactive Similarity Function Assessment

What we have done so far a. Interactive Similarity Function Assessment

What we have done so far a. Interactive Similarity Function Assessment Inter-cluster visual correlation visualization Scalability

Feature Quality Evaluation

What we have done so far a. Interactive Similarity Function Assessment Similarity function is suitable for measuring visual similarity contexts? Similarity function combination (kernel combination) is good? HD projection can precisely preserve original visual similarity contexts? Feature quality is good?

What we have done so far Misleading Effects: Data Uncertainty Junk Images

What we have done so far Duplicates/Near-Duplicates Misleading Effects: Data Uncertainty

What we have done so far Multiple text terms may share similar semantic meaning! One single text term may have multiple semantic meanings! Misleading Effects: Data Uncertainty

What we have done so far Multi-Modal Information Association Misleading Effects: Data Uncertainty

What we have done so far Misleading Effects: Data Uncertainty

What we have done so far b. Interactive Decision Function Evaluation SVM Decision Boundary Visualization & Assessment

What we have done so far b. Interactive Decision Function Evaluation Concept 1Concept 2 GMM Model Visualization

What we have done so far b. Interactive Decision Function Evaluation

Interactive User Involvements via labeling

What we have done so far Updating kernel combinations (kernel weights); Updating projection optimization criteria to preserve similarity better! Updating decision function: margin between positive samples and negative samples! Updating hypotheses for data representation & similarity characterization! b. Interactive Decision Function Evaluation

Enlarge the margin between two classes!

Larger margin has good generalization property!

What we have done so far c. Interactive Knowledge Exploration

What we have done so far c. Interactive Knowledge Exploration

What we have done so far c. Interactive Knowledge Exploration

What we have done so far c. Interactive Knowledge Exploration

What we have done so far d. Collaborative Multimedia Analytics People have strong motivations (self or social) to perform collaborate multimedia understanding!

What we have done so far d. Collaborative Multimedia Analytics

What we have done so far d. Collaborative Multimedia Analytics Learning task and training group organization; Human-computer communication on data and knowledge representation; Human-human communication on hypotheses and knowledge.

More information is available at: Q & A!