1. MIR Lab: R&D Foci and Demos （ MIR實驗室：研發重點及展示）
J.-S. Roger Jang （張智星）
Multimedia Information Retrieval (MIR) Lab
CSIE Dept, National Taiwan Univ.
2018/5/13

2. Our R&D Foci About me Mission Approaches Application domains Music
Use machine learning to tackle real-world problems with immediate applications
Approaches
New learning paradigms
GPU for big data
Application domains
Music
Retrieval and analysis
Speech
Recognition, scoring, and synthesis
Image
Classification and analysis for semiconductor manufacturing automation

3. Music-related Research
Mature technologies
Query by singing/humming
Audio fingerprinting
Music genre classification
Music mood classification
Beat tracking
Query by tapping
Pitch/time modification
Under development
MART
Audio watermarking
Audio melody extraction
Singing voice separation
Score following
Drum id for gaming
Singing scoring
Vibrato detection
Enthusiasism detection

4. Focus: Music Retrieval
Large-scale music search
Query by singing/humming
Audio fingerprinting
Achievements
Top-ranked for some MIREX tasks:
Genre classification
Mood classification
Beat tracking
Audio melody extraction
Technology transfer to several companies
Flow chart
Applications on toys
Video
Clients
Cloud servers
Request:
Acoustic features PC
Smartphones
Response:
search result
Mobile devices
I’m Billy Bass. I know what you are singing!
Pat me and sing to me!

5. Demos for Music-related ResearchPC
Query by singing & humming
Audio fingerprinting
Genre classification
Beat tracking
Singing voice separation
Pitch scaling
Real-time pitch tracking
Drum position id.
Embedded systems
QBSH over Toys
Apps
Auto-rhythm game
Beat-off drum game
I’m Billy Bass. I know what you are singing!
Pat me and sing to me!

6. Speech-related Research
Mature technologies
Voice commands (語音命令)
Speech scoring (語音評分)
Mandarin, English, Japanese, Taiwanese
Text-to-speech synthesis for Mandarin (語音合成)
Speech emotion classification
Under development
Long utterance and text alignment
Speaker recognition

7. Snapshots of ASRA Applications
華語語音評分軟體（授權給資策會）
日語語音評分軟體（授權給巨匠電腦）
英語語音評分軟體（授權給Speak2me公司）

8. Demos for Speech-related ResearchPC
Idiom relay (成語接龍)
Recitation machine (唸唸不忘)
Bricks of idioms (一語中的)
Stress detection（重音偵測）
Text-to-speech synthesis
Chinese conversation classroom
Speech scoring
Voice commands
Lucy’s Café
Embedded systems
Toys
Voice commands over iOS/Android
Mobile apps
Speech scoring game

9. Image-related Research
Projects with TSMC
Wafer map failure pattern recognition
Depth from SEM images
Defect circuit image detection
Wafer image enhancement
Etching width prediction
Face-based analysis
Face recognition
Age estimation
Expression ID
Gender classification
Others
Human identification
Particle tracking
Leaf identification
People counting

10. Thank you for your attention! Questions & comments?