ITCS 6157/8157: Visual Database Jianping Fan Fall 2017 http://webpages.uncc.edu/jfan/itcs6157.html

Overview Class hour 6:30PM - 9:15PM, Wednesday Office hour Wednesday 1:30 - 5:00PM Classroom Woodward Hall 130 Instructor - Dr. Jianping Fan email - jfan@uncc.edu Office – Woodward 205D Webpage http://webpages.uncc.edu/jfan/ Textbook: we will use the slices and papers on the course web page

Why we should have this course? Internet & smartphones are changing the world

Why we should have this course? Multimedia (especially visual data) is dominating the content of Internet

Why we should have this course? Easy access of multimedia content through Internet & mobile devices could be the future of IT

Why we should have this course? This class will help you make these happen! Image/video analysis for feature extraction Database indexing for fast search Machine learning for content understanding Many others…..

Why we should have this course? Good job market: Google, Facebook.... Have fun: solving real problem Not so “hard” to learn (??) Next generation search engines

Course objectives Networks Google, Yahoo! & MSN IE User How can I access multimedia in database over networks? Networks User Multimedia Server

Course objectives 1. How to format multimedia queries? To answer above question, we need to address: 1. How to format multimedia queries? 2. How to represent multimedia content? 3. How to index large-scale multimedia? 4. How to search multimedia in database ? 5. How to transmit query results over IP ? 6. How to control user’s access ? Everyone has smartphone now

Can we do multimedia retrieval like Google for text search? How to build multimedia search engines? Yahoo, Google How to build text indexing? Natural language processing Text document Keywords Text database Inverse File indexing Simple extension multimedia analysis Multimedia data Multimedia ``keywords” Multimedia database & query Hash Indexing or others

Required Techniques How to build multimedia search engines? Computer Vision Technologies for Multimedia Content Analysis Machine Learning Tools for Understanding Multimedia Semantics Database Techniques for Large-Scale Multimedia Indexing Human-Computer Interaction for query formulation, display & exploration

Components from Database System Data Representation Schema Database Indexing Database Storage Query Management Big Data Analytics

Database Indexing Structures

Database Indexing Structures

Components from Computer Vision Image & Video Analysis & Feature Extraction Object Detection & Scene Understanding Classifier Training for object and concept detection Scene Configuration and Structure

Computer Vision

Components from Machine Learning GMM & Bayesian Network Support Vector Machine (SVM) Graphical Models & Structure Learning Statistical Inference Deep Learning & Big Data Analytics

Machine Learning

Database Management System: ITCS6160 or ITCS3160 Computer Vision Pre-Requirements of this Class Database Management System: ITCS6160 or ITCS3160 Computer Vision Machine Learning Programming Skills Willing to work hard If you do not have these background, you should

Course Topics Data Clustering Tools Machine Learning Techniques Multimedia Analysis Technologies Database Indexing Structures Deep Learning & Big Data Analytics Human-Computer Interaction Tools Taking-Home Self-Study Materials Open Discussion & Student Presentation 3

Grading Composition Scale Project 25% (excellent implementation can be up to 45%) Midterm & Final 75% Scale >93% = A 75-93% = B 55-74% = C <55% or cheating = F If you miss 3 classes (three weeks) or more, you are not allowed to take tests (mid-term and final)! 3

Class Policy You have to attend the class & come to classroom on time (no later than 6:35pm) You should be ready to learn from the class: project implementation could be critical You should respect your classmates: come to learn from their presentation!

Classroom Policy No food!!! Drink could be allowed & Cell Phone should be turned off. Small talk is not allowed, but you are welcome to ask questions! Walking inside classroom is not allowed within presentation time!

Course Projects We will offer two kinds of projects: Project implementation project: you need to set up a team or individual to implement one small system for multimedia content analysis or understanding. Paper presentation project: you need to pick one topic to present in the class. MS students are not encouraged to take this kind of project! More information http://webpages.uncc.edu/jfan/itcs6157.html 3

Implementation Project Develop image/video analysis system using Visual C++ and Java. Each group consists 3-4 students 3-4 hours workload each week is expected Java or C++ assumed Talk to your professor to decide which algorithm you may implement for your project, discuss progress with your professor if necessary Demonstrate your implementation to your professor & get feedback 3

Paper Presentation Project If you are PhD student Present one research topic: you need to talk to your professor to get relevant research papers, prepare presentation slides & present in the class. Well-understanding of the topic Good presentation in the class Be able to answer questions from classmates & professor Topic selection: depending on available topics and professor assignment. 3

Course Projects If you do wonderful job on course project, you may expect: Good grade even you may perform well in final and mid-term tests Practical implementation means more than paper work Good recommendation letter for job hunting: professor can only memorize students with good performance! Research position opportunities 3

Midterm & Final closed books and notes Cumulative No makeup One page notes is permitted Cumulative No makeup Bonus is expected Key components for your final grade If you miss 3 classes or more, you are not allowed to take tests (mid-term and final)! 3

Suggestions from Instructor Do your best in the class Show your problems to the instructor when you cannot make it Show the evidence to us if you think you are right. Open discussion is welcome, but no small talk

10-hours Golden Rules 3 hours before class: go through the topics, presentation slides and seek some relevant online documents, …; ready to ask questions in class 3 hours in class: listen to domain experts and try to ask questions 4 hours after class: review what you have learnt from the class, do your project and assignments…

Who cares?

Who cares? Google Search Engine Google Search Engine

Who cares?

The way to join them Good grade from class More training on programming skills, especially for multimedia analysis, indexing and retrieval Get recommendation from professor

Recommendation Good grade is very important, but it is not everything! Learning something and solving one problem you like may be more important! Learning from someone who may make you better! Especially your classmates

What areas we will touch? Computer Vision Database Information Retrieval Machine Learning & AI Visualization Networks Statistics & Security

Start-up Companies What you may expect Many wonderful companies & start-ups come from course projects! You could be the next one!

Start-up Companies What you may expect Product search engine for amezon.com, taobao.com Using your smartphones to take pictures, then we will find the cheapest ones for you!

What you may expect Start-up Companies Image Search Engine

Start-up Companies What you may expect Google Glass App: Google glass may change world like i-phone

Start-up Companies What you may expect Digital Camera App: Sony may sale digital cameras with your media organization & search software.

Start-up Companies What you may expect Personal Computer App: IBM Dell may sale PCs with your media organization & search software.

Start-up Companies What you may expect Automatic-Driving Car App: BMW Tesla may sale cars with your object recognition & navigation systems.

Start-up Companies What you may expect Plant Species Identification: Your Kids will be proud of you because you Know every plant species on the world

Start-up Companies What you may expect Plant Species Identification: Your Kids will be proud of you because you Know every plant species on the world

Start-up Companies What you may expect Construction Safety Alarm: educators, government & insurance companies may care

Start-up Companies What you may expect Multimedia Search Engine: Google will definitely care!

What I or UNCC may expect Do not forget to come back UNCC & support our research!

why not ask "stupid" questions? Do your best & have fun! Good students should be able to push your professor to think and work harder not easier!

I am a nice professor if you do your jobs!