NM7613: Music Signal Analysis & Retrieval (音樂訊號分析與檢索)

Roger Jang (張智星)

Schedule of Spring 2017

[Week1] [Week2] [Week3] [Week4] [Week5] [Week6] [Week7] [Week8] [Week9] [Week10] [Week11] [Week12] [Week13] [Week14] [Week15] [Week16] [Week17]

The Schedule will be changed as we move on.

Agenda:
1. Intro. to the course (Slides, recording-20160223)
2. Demos of speech/audio applications by MIR lab
  - Music-related demos on PC:
    1. Miracle (for query by singing/humming)
    2. Audio fingerprinting
    3. Genre classification
    4. Beat tracking
    5. pitch scaling demo
  - Speech-related demos on PC:
    1. MIR TTS (for text-to-speech conversion)
    2. Recital machine（唸唸不忘） (based on speech assessment)
    3. Speech-enabled Chinese idiom relay（成語接龍） (based on speech recognition)
    4. Speech-enabled Chinese idiom riddle（一語中的） (based on speech recognition)
    5. Voice command: Chinese, English, Japanese, Taiwanese
    6. Speech scoring: Chinese, English, Japanese, Taiwanese
  - Demos on Embedded systems and toys:
    1. AutoRhythm game (youtube)
    2. Singing fish/dog (大嘴鱸魚和唱歌狗) with melody recognition (youtube, mir)
    3. Robot fighter (蘿蔔戰士) with speech/melody recognition, Wii remote control (youtube)
    4. Chicken run（落跑雞）with English speech recognition (youtube, mir)
    5. Tang poem recital penguin（唐詩背書企鵝）with Mandarin speech recognition (mir)
Assignments:
1. Join the FB page of this course
2. Update your secondary email address on CEIBA
3. Install MATLAB on your computer (Install tutorial for CSIE/NTU)
4. Install Audacity, GoldWave, or CoolEdit
Take-home reading:
- Read either of the following (make sure you try out all the examples):
  - MATLAB Primer (in English)
  - "MATLAB小傳與外觀" and "初探MATLAB" at MATLAB程式設計《入門篇》 (in Chinese)
No class due to 和平紀念日
Agenda: MATLAB programming
1. MATLAB programming: Please check out the following topics at "MATLAB程式設計《入門篇》".
  - 02-初探MATLAB (Intro to MATLAB): Recordings 20170306, 20160301
  - 03-二維平面繪圖 (2D plotting): Recordings 20170306, 20160301
  - 11-多維陣列 (Multi-dimensional arrays): Recordings 20160301
  - 12-異質陣列 (Cell arrays)
  - 13-結構陣列 (Structure arrays)
2. Hints to homework.
Assignments:
1. Function for computing the Fibonacci number (You only need to do part (b).)
2. Sorting and its inverse
3. Reorganize a 3D array
4. Find statistics of songs
Take-home reading:
- Watch the video of "音訊讀寫、錄製與播放" (in Mandarin) at MATLAB程式設計《入門篇》.
- All the examples and text are in Chapter 4 "MATLAB for Audio Signal Processing" of the online tutorial Audio Signal Processing.
Agenda:
1. Introduction to Audio Signals: Slides and recording-20160308
2. MATLAB for Audio Signal Processing:
  - Chapter 20 of MATLAB程式設計《入門篇》: Slides and recording-20160308
  - Chapter 4 of Audio signal Processing and Recognition
Assignments:
1. Generate sine wave with time-varying frequencies
2. Voice signal encryption
Take-home reading:
- Audio signal Processing and Recognition: Basic Acoustic Features of Audio Signals (You only need to read 5-1 to 5-3).
Agenda:
1. Audio signal Processing and Recognition
  - Basic Features of Audio Signals: Slides and recording-20160315 (Unfortunately, it's mute from 16:55 to 44:10 due to malfunction of recording.)
2. Explain how to do recording and labeling of EPD.
Assignments:
1. Zero justification via polynomial fitting
2. Recordings of digits and letters (These recordings are to be used for the EPD programming contest.)
Take-home reading:
- Audio signal Processing and Recognition: End-points Detection (You only need to read 6-1 to 6-2).
Agenda:
1. Audio signal Processing and Recognition
  - End-points Detection: Slides and recordings ( 20170327-intro, 20170327-volume, 20170327-zcr, 20170327-hod, 20170327-spectrum, 2015)
2. Explain how the program contest works.
3. Explain the homework.
Assignments:
1. Programming contest: end-point detection (Due at 20170409 midnight)
Take-home reading:
- Audio signal Processing and Recognition: Pitch Tracking (You only need to read 7-1, 7-2, and 7-4).
Spring break
Agenda:
1. Audio signal Processing and Recognition
  - Pitch tracking in time domain: Slides, recordings ( 20170411, 20160412, 2015)
2. Explain the basic components of my m-file functions.
3. Explain how to do pitch labeling.
4. Play the highest and lowest pitch
Assignment:
1. Recording task: Children's songs (Note that the your labeled pitch will be examined by my program, so make sure you do a decent job on pitch labeling! Please ask if you have any questions during your labeling.)
2. Programming contest: Pitch tracking (Due at 20170507 midnight.)
Take-home reading:
- Audio signal Processing and Recognition: Pitch Tracking (frequency domain).
Agenda:
1. Audio signal Processing and Recognition
  - Discrete Fourier transform: Slides and recordings ( 20170417, 20160418, 2015)
  - Frequency-domain pitch tracking: slides and recordings ( 20170417, 20160418, 2015)
  - Pitch tracking - pre and post processing: Slides
  - Pitch tracking - performance evaluation: Slides and recordings ( 20160418)
Assignment:
1. Programming contest: Pitch tracking (Due at 20170507 midnight.)
Take-home reading:
- Audio signal Processing and Recognition: Linear scaling in Chapter 14, Query by Singing/Humming
Agenda:
1. Audio signal Processing and Recognition: Chapter 14: Query by Singing/Humming
  - Introduction to MIR: Slides and recordings ( 20170424, 20160426, 2015)
  - QBSH retrieval methods: Slides and recordings ( 20170424, 20160426, 2015)
  - Midterm evaluation of MSAR: Slides
  - More about the programming contest of pitch tracking.
Assignment:
1. Please choose one of the following tasks:
  - Play a song (for instance, "happy birthday to you") to TA using any whistles other than the normal one, which includes wolf whistle, hand whistle, fingerless whistle, leaf whistle, etc. (Demo required)
  - Recording of English multi-syllable words (for word stress detection): Please download the English word recorder from this page.
Take-home reading:
- Audio signal Processing and Recognition: DTW (types 1, 2, and 3) in Chapter 14, Query by Singing/Humming
Agenda:
1. Data Clustering and Pattern Recognition: Chapter 8: Dynamic Programming
  - Intro to Dynamic programming (DP): Slides and recordings ( 20170501, 20160503, 2015)
  - Longest common subsequence (LCS): Slides and recordings ( 20170501)
  - Edit distance (ED): Slides and recordings ( 20170501)
  - Matrix chain product (MCP): Slides
2. Presentation of 3 winners for EPD programming contest.
Assignment:
1. Dataset collection for audio fingerprinting (Due in a week)
Agenda:
1. Audio signal Processing and Recognition: Chapter 14, Query by Singing/Humming
  - Dynamic time warping (DTW): Slides and recordings ( 20170508, 20160510, 2015)
  - DTW for QBSH: Slides and recordings ( 20170508, 2015)
Assignment:
1. Programming contest: query by singing/humming (Due date: to be announced later)
Agenda:
1. Onset detection: Slides, recordings ( 20170515, 20150514)
2. Details of the assignment of beat tracking
Assignment:
1. Beat tracking: Please read readme.txt after downloading. (Due date: to be announced later)
Agenda:
1. Audio fingerprinting: Slides & recordings ( 20150520, 20170522)
2. Presentation of winners of our previous assignment (pitch tracking)
3. More about the contest of QBSH
Assignment:
1. Programming contest: query by singing/humming (Due date: 20170604 midnight)
2. Recording task: Youtube pop song recording (For students of native Chinese speakers only.)(Due date: 20170528 midnight)
No class due to Dragon Boat Festival.
Agenda:
1. Introduction to MIREX/ISMIR, recordings ( 20170605)
2. Music genre classification
  1. My introduction, recordings ( 20170605)
  2. Brian's slides for 2012 ICMLA
3. Singing voice separation: slides, recordings ( 20170605)
Assignment:
1. Programming contest: Beat tracking (Due at 20170625 midnight.)
Agenda:
1. WSOLA: slides
2. Closing remarks: slides
Assignment:
1. Programming contest: Beat tracking (Due at 20170625 midnight.)