Presentation is loading. Please wait.

Presentation is loading. Please wait.

CS335 Principles of Multimedia Systems Cameras and Projectors Hao Jiang Computer Science Department Boston College Oct. 2, 2007.

Similar presentations


Presentation on theme: "CS335 Principles of Multimedia Systems Cameras and Projectors Hao Jiang Computer Science Department Boston College Oct. 2, 2007."— Presentation transcript:

1 CS335 Principles of Multimedia Systems Cameras and Projectors Hao Jiang Computer Science Department Boston College Oct. 2, 2007

2 CS335 Principles of Multimedia Systems Cameras and Projectors  Cameras and projectors have been intensively used in many different multimedia applications.  It is important to understand techniques to manipulate these devices.  We will study basic methods about –calibration, –image warping and blending, –and other issues in building a camera/projector system.

3 CS335 Principles of Multimedia Systems Example Application: Projector Geometry Distortion Compensation Automatically correcting projector geometry distortion using a camera and projector system.

4 CS335 Principles of Multimedia Systems Camera / Projector Geometry Model  Camera and projector can be modeled as a pinhole imaging system. Object Point Optical Center Image Plane Image point Focal length Optical axis

5 CS335 Principles of Multimedia Systems Camera Geometry Model  A camera projects a 3D point onto a 2D point in a image. (X’,Y’,Z’) x = (f X’/Z’)/dx + O x = f x X’/Z’ + O x y = (f Y’/Z’)/dy + O y = f y Y’/Z’ + O y in camera’s frame is dx and dy are width and height of an image pixel f (O x, O y ) 3D point (X, Y, Z) x y

6 CS335 Principles of Multimedia Systems Homogenous Coordinate  The homogenous coordinate of a 3D point (x,y,z) is (X,Y,Z,W) where X/W = X, Y/W=Y and Z/W=Z.  The homogenous coordinate of a 2D point (x,y) is (X,Y,W) where X/W = x, Y/W=Y.  We can convert a 3D point from one frame to another, by simply T*P, where T is a 4x4 matrix and P is the homogeneous coordinate of the 3D point.

7 CS335 Principles of Multimedia Systems Camera Matrix  Using homogenous coordinate, the camera projection can be represented as x y w ~ A 3x4 X Y Z W Where A 3x4 is a 3 rows and 4 columns matrix, called camera matrix. p = = A P a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 P =

8 CS335 Principles of Multimedia Systems Projection from a Plane to Another Plane O p P p = A P X Y Z SinceWe have xywxyw ~ a 11 a 12 a 14 a 21 a 22 a 24 a 31 a 32 a 34 XY1XY1 = p =H p’ H is defines a Homography.

9 CS335 Principles of Multimedia Systems The Projector Model  Projector shares similar model with a camera.  The only difference is that a projector projects a flat object that emits lights based on a computer image onto an image plane, the screen.  Based on the previous analysis about planar object projection, the image from the computer and the one projected on the screen are related by a homography.

10 CS335 Principles of Multimedia Systems The Camera-Projector System Image sent to projector Camera image Image on the screen H1*pH1*p H2*rH2*r p q r H 2 *H 1 *p

11 CS335 Principles of Multimedia Systems Calibration  We would like to compute H1 and H2.  We project marker points on the screen and form equations a 11 X n + a 12 Y n + a 14 a 31 X n + a 32 Y n + a 34 a 21 X n + a 22 Y n + a 24 a 31 X n + a 32 Y n + a 34 = x n = y n n = 1.. N a 34 =1

12 CS335 Principles of Multimedia Systems Image Adjustment (I) Projector image Camera image Image on the screen H1*pH1*p H 2 *H 1 *p Pre-warping p q Each p is projected to the camera image and the color is determined by color interpolation in the desired image. Assume that the viewer is close to the camera. The desired Image.

13 CS335 Principles of Multimedia Systems Image Adjustment (II) Project image Camera image Image on the screen H1*pH1*p H 2 *H 1 *p Pre-warping Estimate H 2 using screen corners projections. Pre-warp image based on the desired image on the screen. The desired image

14 CS335 Principles of Multimedia Systems Projection onto Arbitrary Surfaces  We have learned how to deal with projector distortion using a planar screen.  We can extend the method into other types of surfaces, such as cylinder or spheres.  We need a relative dense mesh grid to capture the local deformation model.

15 CS335 Principles of Multimedia Systems Titled Large Screen Display

16 CS335 Principles of Multimedia Systems Titled Large Screen Display  LCD or CRT are still not easy to be made as large as a wall.  Titling multiple projector images into a large screen display is flexible and relatively cheap.  The shortcoming is we need to align the images from different projectors in both geometry and color.  Manually adjusting the projectors is a tedious task.  Camera project system can be used to solve the problem.

17 CS335 Principles of Multimedia Systems Titled Images Projector images The camera view Calibration Patterns (Projector to Camera Homography can be computed based on these patterns)

18 CS335 Principles of Multimedia Systems Large Screen Projection Projector images The camera view The big image p camera = Hq projector p q 3 projector image

19 CS335 Principles of Multimedia Systems Edge and Color Alignment  Colors of different projectors are usually different. A color calibration and adjustment procedure is needed.  Blending 2 3 Color(p) =  Color(2,p) + (1-  )Color(3,p) p  is determined by the dominance of 2 or 3.

20 CS335 Principles of Multimedia Systems Blending  The size of the blending region has to be carefully selected  If the region is too small, large scale structures will show abrupt changes. If it is too big, small structures (edges, dots) will overlap in a big region and therefore will result in blur (ghost) images.  1-  Blending region

21 CS335 Principles of Multimedia Systems Multi-band Blending  Multi-band blending can be used to address the problem.  In multi-band blending, –images are filtered into different bands. –The mask is also low-pass filtered to generate mask for each band images. –Images are blending in each channel. –The blended images are summed up to get the final result.

22 CS335 Principles of Multimedia Systems Image Pyramids The Gaussian Pyramid

23 CS335 Principles of Multimedia Systems Image Pyramids The Laplacian Pyramid

24 CS335 Principles of Multimedia Systems Image Pyramids The Gaussian pyramid of the mask

25 CS335 Principles of Multimedia Systems Multi-band Blending Multi-band blending

26 CS335 Principles of Multimedia Systems Multi-band Blending

27 CS335 Principles of Multimedia Systems System Issues of Large Screen Display  Large screen projection needs multiple projectors working simultaneously. –The first structure uses share memory system such as SGI Oynx2, that employs a shared-memory model. A single program can have different threads writing OpenGL primitives into different pipes while reading from a single shared database and synchronizing display update over shared flags. –The second structure is PC cluster, in which each PC handles one projector. This framework is much cheaper but the synchronization is a challenge problem.

28 CS335 Principles of Multimedia Systems Applications  Visualization and Collaboration IEEE Computer Graphics and Applications, 2000

29 CS335 Principles of Multimedia Systems Immersive Reality  The CAVE (University of Illinois at Chicago) IEEE Computer Graphics and Applications, 2000

30 CS335 Principles of Multimedia Systems Projection for Augmented Environment Anton Treskunov and Jarrell Pair, PROJECTOR-CAMERA SYSTEMS FOR IMMERSIVE TRAINING, ASC06

31 CS335 Principles of Multimedia Systems Input Methods for Large Screens  It presents challenges for traditional input methods.  Possible HCI methods include: –Pointing devices, including 3D tracking, passive optical (video) tracking, ultrasonic tracking, mice, and tablet interfaces; –User tracking, for point-of-view rendering or for gaze directed interaction, via optical tracking or electromagnetic tracking; –Handheld devices, providing control interfaces that can be out of band from the display; –Voice commands with audio feedback; –And haptics interfaces.

32 CS335 Principles of Multimedia Systems Multiple Camera System  Multiple camera system can capture video from different locations simultaneously.  Stereo system has been widely used for inferring the “depth” of objects.

33 CS335 Principles of Multimedia Systems 3D reconstruction from Multiple Views

34 CS335 Principles of Multimedia Systems Application of Multi-camera Imaging  Image based rendering  The Matrix

35 CS335 Principles of Multimedia Systems Cameras in the Matrix

36 CS335 Principles of Multimedia Systems Camera Calibration Toolboxes  OpenCV  Matlab Calibration Toolbox –http://www.vision.caltech.edu/bouguetj/calib_doc/http://www.vision.caltech.edu/bouguetj/calib_doc/  Multiple Camera Calibration –http://cmp.felk.cvut.cz/~svoboda/SelfCal/


Download ppt "CS335 Principles of Multimedia Systems Cameras and Projectors Hao Jiang Computer Science Department Boston College Oct. 2, 2007."

Similar presentations


Ads by Google