1. Savant Tony Wu – Jay Ni – Deri Kusuma Automated music improvisation 1 Special Thanks to: Prof. Plummer and Cristina Pop and our testers: Tim Lambert, Adriana Miu, Jon Lau, and You Li

2. Architecture 2

3. Algorithms 3

4. Music Representation 4 - Screenshot of Cakewalk Music Creator 3 Note Time Pitch

5. Note Representation 5 Struck time Pitch Release time Velocity = struck volume Time Volume (db)

6. Music Timing 6 =120 @ 120 beats per minute 1 tick 125 ms 1 beat 0.5 sec 1 measure Tempo (beats per minute) @ 4 ticks per beat

7. Logic Architecture 7 GUI Input buffer Libraries and intermediate data structures Output buffer MIDI controller MIDI input file Input manager Decision Engines Analyzers Improvisers Output manager Synthesizer MIDI output file Logic thread Input thread Output thread Visualization Audible sound GUI thread

8. Real-time Music Storage 8 Circular chunklist Press noteO(1) Release noteO(1) Range queryO(A + n) Time overheadO(n) per tick Memory usageO(NL) A: number of answers n: avg. number of notes per tick N: total number of notes stored L: avg. length of notes in tick

9. Chord analysis – basic chord types 9 These are like basic colors in painting Other chords are simply enrichment / inversion / transposition of these basic feels majorminordimaugdomhalf dimsus4C: EnrichmentInversionTransposition(of C major)

10. Chord analysis – note domains 10 Important for generation! C: Id: VIIbG: IV Affinity to A, B Distance from F, Ab Affinity to F, AbAffinity to F# These are all C major chords:

11. Chord Analysis – Note Desirability 11 C major: Only C,E,G will sound good down here F and F# only sound good with high octaves D,A,B sound good if not too low Higher pitch has less chord determining capability

12. Chord Analysis – Capturing Richness 12 G dominant 7 b9 b13 G F B Eb Ab Bass Strong notes Higher notes Storing the note distribution of input captures richness beyond basic types (e.g. major, minor) C, Db, D, E, F#, A, Bb Absent

13. Chord Analysis – Capturing Ambiguity 13 We compute a score for each possible chord interpretation, and then pick the highest score to determine chord. CDbDEbEFF#GAbABbB M.93………….50……………… m.47…………………….70…… dim-.44…………………….34…… aug-.76…………………………… dom.23…………………………… halfdim-.45…………………….48…… …………………………………

14. Decoupling Motif and Chord 14 C major G 7 + + + Bb minor 7 9 / F = = = The same motif

15. Offline Motif Fitting 15 C B Bb A Ab G F# F E Eb D Db C B Bb A Ab G F# F E Eb D Db C One-to-one pitch mapping algorithm with dynamic programming Maximize objective function: REWARD chordal fitness as a function of motif and destination chord PENALTY absolute displacement, relative displacement TIME BUDGET 60-120 ms per motif

16. Real-time Motif Fitting 16 Pitch Time Maximize objective function per assignment, given assistance level REWARD chordal fitness as a function of destination chord, consistency to assignment history PENALTY absolute displacement, collision with assignment history for neighboring notes TIME BUDGET < 1 ms per note The present input mapped

17. Determining Key of Piece 17 HOW? The current key is determined by the discrete convolution of the input notes with each major / minor filter. Performing Key Changes Abrupt: Using dominant chord of new tonic Smooth: Leverage current chord progression C: I – V – vi – iii – F:I – V – vi - … (C:IV – I – ii - …)

18. Pleasant Chord Transitions 18 Chord State MachineChord type heuristic IM, M7, M9 IIm7 IIIm, D7 IVM, M7, m VD7 VIm, m7 VIIdim, dim7, halfdim7 (From music theory)

19. Chord Phrasing 19 Forcing a phrase to end in a chord (e.g. V7) produces clearer phrase boundaries. C: I – iii – vi – IV – ii – V7 – I – V7 If this isnt possible… C: I – iii – vi – IV – ii – I – iii – V7 Take 2 steps back! C: I – iii – vi – IV – ii – – – V7

20. Savant Modes 20 ModeDescription Co-opDuet with human on harmony, human on melody. Melody is fitted to match the harmonic progression. AI Co-opDuet with AI on harmony, human on melody. Melody is fitted to match the harmonic progression. RemixAnalyzes motif and chords of input, and generates a remixed version with motif and chord substitutes. User can play along and choose to influence direction of generation, or to have input fitted to chord progression. Call & ResponseUser plays a musical phrase, and computer will respond with an equal length phrase. EmptyNo logic, just like a real piano!