Download presentation

Presentation is loading. Please wait.

Published byKya Bones Modified about 1 year ago

1
Computational Photography and Capture: Time-Lapse Video Analysis & Editing Gabriel Brostow & Tim Weyrich TAs: Clément Godard & Fabrizio Pece

2
Today’s schedule Computational Time-Lapse Video – Bennett & McMillan, Siggraph 2007 Factored Time-Lapse Video – Sunkavalli, Matusik, Pfister, Rusinkiewicz Siggraph 2007 Video Synopsis & Indexing, – Pritch, Rav-Acha, Peleg, ICCV 2007

3
Time-lapse Photography Definition: when frames are captured at a lower rate than the rate at which they will ultimately be played back. Compare to – Slow-motion – Bullet-time

4
Time-lapse Filmmaker’s Challenges Lighting changes (strobing) High frequency motion (missed action) – Saturation & exposure Camera motion: intentional or not Triggering / data storage / camera safety Useful? Editable?

5
Does the old definition of time-lapse still hold?

6
(Quick Overview of) Computational Time-Lapse Video Bennett & McMillan Siggraph 2007 (Project web page) (though link seems to have died, so see my archive here)here

7
Virtual Shutter

8

9
Sampling in 1D Uniform Interpolate linearly+ piece-wise vs. Same # of samples, but where needed! Optimum polygonal approximation of digitized curves, Perez & Vidal, PRL’94 – Use Dynamic Programming to find global min.

10
Video

11
Completely User-controllable: Min-Change, Min-Error, Median, etc

12
Factored Time-Lapse Video Sunkavalli, Matusik, Pfister, Rusinkiewicz SIGGRAPH 2007 Project web page

13
Remember Intrinsic Images from Video? Weiss ICCV’01

14
Outdoors: More Than Just Color vs Illumination? OriginalNo Shadows Given: Daytime, under clear-sky conditions Wishlist Remove shadows Render new shadows Change albedo Change global illumination Wishlist Remove shadows Render new shadows Change albedo Change global illumination

15
Outdoors: More Than Just Color vs Illumination? OriginalNo Shadows Lighting due to sun? sky? Surface Normals?

16
Formulation size: width x height x time F : xyt volume of frames over time I sky : Accumulated intensity due to sky-light I sun : Accumulated intensity due to sun-light S sun : Binary; is a pixel out of shadow?

17
Intuition

18

19
Separate Sky first Photoshop! – Just to pick “non-surfaces” When is sun’s contribution == 0? – Leaving just sky’s contribution

20
“Definitely” shadow: median of darkest 20%, then threshold at 1.5x

21
Bilateral Filter “Definitely” Shadow

22

23
How Did Skylight Change? I sky : Accumulated intensity due to sky-light W sky : Sky-light image H sky : Sky-light basis curve (1D function!!) – Whole scene got brighter/darker together

24
Blue: skylight curve (same one)

25
Factorization Decompose appearance into – per-pixel W – H curve Alternating Constrained Least Squares (ACLS): – H (t) is held fixed while W is optimized using least squares, then vice versa

26
ACLS: Alternating Constrained Least Squares Inverse shade trees for non-parametric material representation and editing, – Lawrence et al., Siggraph 2006, code online Lawrence et al., Siggraph 2006online

27

28
Original and I sky

29
I sky and Reconstruction

30
So Far - Explained Sky

31
Now Decompose Sun’s Contributions I sun : Accumulated intensity due to sun-light W sun : Sun-light image, per-pixel weights H sun : Sun-light basis curve : Shift-map; per pixel offset in time

32

33
I sun : Accumulated intensity due to sun-light W sun : Sun-light image, per-pixel weights H sun : Sun-light basis curve : Shift-map; per pixel offset in time

34
I sun : Accumulated intensity due to sun-light W sun : Sun-light image, per-pixel weights H sun : Sun-light basis curve : Shift-map; per pixel offset in time

35
Total sunlight contribution Sunlight image (like on moon) Shift mapSunlight

36
Factorize Again! For each image, subtract off I sky (t) S sun : Sky-light mask, serves as C, confidence map ACLS again – But add 3 rd update phase, searching for that minimizes reconstruction error of scaled + offset H(t) vs. F(t)

37
Green: sunlight curve (same one, but time-shifted)

38
I sun (t) W sun (time offset)

39
Reconstruction

40
Does It Work? (black curves are calculated sunlight basis curves)

41
Video: FactoredTimeLapseV

42
What have we gained? Can – Remove shadows – Change albedo – Change amount of global illumination Compression Render new shadows: – We have 1 component of the per-pixel Normals, N

43
Estimated Normals (1D)

44
Manipulating Normals

45
(View Results of) Video Synopsis & Indexing Pritch, Rav-Acha, Peleg ICCV 2007, PAMI 2008 (Project web page)

46
What to do with blob-tracks? (Project web page)

47
HDR Time-Lapse Essentially started in 2006 (manually at first)manually at first More are popping up online More

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google