1. Automated HDR Image Capturing System Angela Buttrick, Nate Naber, John Olender

2. Agenda Project Overview System Description User Interface Operating Boundaries Feasibility/Potential Problems Test Strategy

3. Project Overview HDR image – capture a much wider dynamic range of intensities than a regular LDR image

4. Project Overview 1. Remote Control Unit (RCU) takes pictures 2. Pictures are then uploaded to the PC 3. Software package then handles extracting images and converting each series of pictures into HDR images 4. These HDR images run through an algorithm to produce a final picture

5. System Description - Camera Still Image: JPEG (Exif 2.2 compliant)Image Compression Auto/ ISO 50/ 100/ 200/ 400ISO Sensitivity 15 - 1/1,500 sec.Shutter Speed /2.8/ 7.1 (W), f/3.9/ 10 (T)Maximum Aperture 3.2x Digital Zoom (Max 6.4x digital zoom when combined with optical zoom) Digital Zoom 5.4 - 10.8mm, f/2.8 - 3.9 zoom lens (35mm film equivalent: 35 - 70mm)Focal Length High Resolution 3.2 Megapixel CCDMegapixels

6. System Description - RCU Remote Control Unit (RCU): –HCS12 microcontroller –USB host controller –Buttons/LEDs for control and system status –RCU will include a standard- size Type-A female USB port, provided by the USB host controller –Battery four AA rechargeable batteries used to power all contained components.

7. System Description – USB Host Controller MCU-to-USB host controller module, specifically designed for adding USB host functionality to microcontrollers Allows the camera to recognize the RCU as a USB host Makes the USB protocol transparent to the HCS12 FTDI VNC1L MCU to USB Host Controller (VDIP1 Module)

8. VDIP1 Schematic

9. System Description – PTP Picture Transfer Protocol (PTP) – used for communication between HCS12 Canon Camera Canon cameras use a series of command and response blocks to send packets back and forth

10. PTP Send / Receive Packets 0000000: xx xx xx xx yy yy yy yy - 00 00 00 00 00 00 00 00 0000010: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 0000020: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 0000030: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 0000040: 02 00 00 00 UU 00 cc VV - xx xx xx xx SS SS SS SS 0000050:... (payload/arguments)‏ 0000000: 00 00 00 00 yy yy 00 00 00 00 00 00 00 00 00 00 0000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000040: 02 00 00 00 UU 00 cc VV - xx xx xx xx SS SS SS SS 0000050: kk kk kk kk

11. System Description – HDR Software Packages Pfscalibration: allows for the calibration of the camera used to take the pictures and the recovery of HDR images from the multiple exposure images collected using the RCU Pfstools: provides a set of tools for reading and writing HDR images. This will allow for the storage and previewing of raw HDR images Pfstmo: allows for tone-mapping operators to be applied to HDR images, producing images which may be displayed on a LDR display device contains the gradient domain compression scheme tone mapping operators

12. Pfstmo Examples Mantuik '06 Tone Mapping OperatorFattal '02 Tone Mapping Operator

13. User Interface Clean, easy-to-use UI – –Allows user to calibrate his/her camera for optimal HDR images – –Allows user to select a range of images to be processed – –Converts LDR->HDR image by invoking pfstools, pfscalibration, and pfstmo – –Displays resulting image to the user

14. Operational Boundaries Limiting Factors –Portability – Camera can only extend up to 3 meters from the RCU –Image Quality – Necessary to have stable Camera and still image –Power – Limited by battery power, batteries increases portability Four AA batteries can supply ~4.8V. 1800mAh to each component in the RCU requiring power Typical operating conditions: –LEDs draw 12mA (one LED lit)‏ –Buttons draw 12mA –HCS12 microcontroller draw about 100mA –VNC1L USB host controller draw about 25mA –Total current draw of 149mA –~12 hours of use with constant use Battery condition and temperature

15. Analytical Component Time for HDR Automated Flow: –Experimentation to derive time to save image to flash, shutter speed’s effect on capture time Number of photos taken per HDR –Experimentation of the optimal number to produce best image –Affects total memory available –Max # of photos is Dictated by mega pixel size/memory card size

16. Analytical Component Why do we need so many exposures? Range of exposures too narrow (missing very light / very dark information from the scene)‏ Not enough exposures to cover the range (light/dark details in the scene emphasized too highly)‏

17. Feasibility/Potential Problems Controlling shutter speeds via the RCU PTP/IP and Canon PTP User Interface: Compiling set of images into a single HDR image Completely transparent to the user Invoking processes through a GUI and piping I/O quietly Open-source software tools with the software UI Drop-in USB Host Controller Battery Power Low-power circuitry as well as low-power modes WAIT and STOP power-saving modes of HCS12

18. Testing Strategies – UI/Software UI needs to start up properly + allow user to select a number of images only UI needs to start up properly + allow user to select a number of images only Camera calibration function must be tested with the ability to load/save camera calibration info Camera calibration function must be tested with the ability to load/save camera calibration info Image processing must be tested with a list of images loaded and camera successfully calibrated Image processing must be tested with a list of images loaded and camera successfully calibrated Fail-over testing will be performed in depth in order to prevent a number of user errors from occurring Fail-over testing will be performed in depth in order to prevent a number of user errors from occurring

19. Testing Strategies - RCU Setup Setup Layout Physical Connections Layout Physical Connections Establish/Verify Canon Capture Establish/Verify Canon Capture Phase 1 Phase 1 Verify Camera is recognized by PC Verify Camera is recognized by PC Establish/Verify Canon Capture camera connection to PC Establish/Verify Canon Capture camera connection to PC Send simple instruction to Camera, i.e. Zoom 2X Send simple instruction to Camera, i.e. Zoom 2X Check for completion Check for completion Send shutter control from PC Send shutter control from PC Take pictures at various shutter speeds Take pictures at various shutter speeds Remove/Download/Verify Images used different shutter speeds Remove/Download/Verify Images used different shutter speeds Phase 2 – Same as Phase 1 but with the HCS12 instead of the PC Phase 2 – Same as Phase 1 but with the HCS12 instead of the PC Phase 3 Phase 3 Verify Buttons properly send instructions to the Camera Verify Buttons properly send instructions to the Camera Verify LEDs show correct status during HDR capturing Verify LEDs show correct status during HDR capturing

20. Questions?