1. NETWORK RC CAR © James Crosetto BS (Computer Science and Computer Engineering) Jeremy Ellison BS (Computer Engineering) Seth Schwiethale BS (Computer Science)

2. Member Bio’s James Crosetto Computer Science and Computer Engineering Pizza Guru Seth Schwiethale Computer Science Music Diversity Contributor Jeremy Ellison Computer Engineering Anti-Coldplay

3. Presentation Outline Objectives Goal Setting Innovate & Design Requirements Necessary Equipment Design and Implementation Code Review Future Development Where to go from here ObjectivesRequirementsDesign & ImplementationFuture Development

4. Objectives Goal Setting Project Overview Functional Objectives Learning Objectives Innovative development Radio Frequency vs. Internet Computer vs. IP Camera Project OverviewFunctional ObjectivesLearning ObjectivesRF v. InternetAnimated Design ObjectivesRequirementsDesign & ImplementationFuture Development

5. Project Overview Expanding the range of a remote control car Control car over network Be able to have first person view of car’s location Build something cool Objectives Goal Setting ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional ObjectivesLearning ObjectivesRF v. InternetAnimated Design

6. Functional Objectives Establish connection between RC car and driver’s computer Get real time visual feed Control RC car User friendly GUI Objectives Goal Setting ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional Objectives Learning Objectives RF v. InternetAnimated Design

7. Learning Objectives Understand Wireless Communication Efficient methods of sending and receiving data Embedded Systems Microprocessors Assembly Language Linux / C R/C car design and functionality Servos Pulse Width Modulation Objectives Goal Setting ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional Objectives Learning Objectives RF v. InternetAnimated Design

8. Radio Frequency 350-400ft Line-of-sight Object interference Internet Network Advanced Range Non-line-of-sight Objectives Innovative Development ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional Objectives Learning Objectives RF v. InternetAnimated Design

9. Objectives Animated Design Progression ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional Objectives Learning Objectives RF v. InternetAnimated Design

10. Requirements Necessary Components Car Camera Microprocessor User’s PC ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

11. Requirements The Car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

12. 3 Leads: Ground V cc Pulse width modulation Requirements The Car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

13. Requirements The Car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

14. Square Pulse wave of 1.0-2.0ms repeats every ~20ms Width of pulse determines the position of the servo with 1.5ms as the normal center The amplitude of the pulse is from the reference level to the V cc V cc = 4.6-6.0V Requirements The Car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

15. Power: 5.1 V DC, max 3.5 W Alarm output (motion, audio, external) Open API for software integration CPU, video processing and compression; Ram: 32MB Flash: 8MB Requirements IP Camera ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

16. Receives and translates signals from the transistor output of the IP camera Sends translated signals to steering box and speed control Programmed with C and Assembly Requirements Microprocessor DragonFly12 ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

17. Requirements product research – end user View video stream from camera Send commands to the car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

18. Unexpected outcome of design… Swarmed with applications to be our advisor:

19. Project Costs: Camera One time Purchase: $275 Little Caesars Hot-N-Ready: $600 annual cost Traveling Costs Seminar in Hawaii - $800/member Still pending department approval…

20. Implementation & Design ObjectivesRequirementsDesign & ImplementationFuture Development

21. PARTYING? How to accomplish our goals?

22. NO!!!

23. STUDYING?

24. YES!

25. Implementation 3 Major Lines of Communication PC  Camera Camera  Microprocessor Microprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

26. 1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  CameraCamera  MicroprocessorMicroprocessor  Car Implementation Animated Diagram ObjectivesRequirementsDesign & ImplementationFuture Development

27. Design: PC to Camera Software requirements to communicate Server to take commands (camera side) Client to give commands (user pc side) GUI to show video feed Reusability PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

28. Server Background ARTPEC processor running Linux 2.6.18 on a armv4tl architecture Server written in C Compiled using an x86 to ARM cross- compiler PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

29. Crashing Ubuntu v6.06 LTSv6.10 V8.10v8.04 v7.10v7.04 ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

30. Server Implementation TCP/IP Socket can be thought of as a file Basic Process create socket bind to port listen on port for connections accept connection handle input PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

31. Server code ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

32. Server implementation ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

33. Server implementation ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

34. Client Background Java for cross-platform TCP/IP Socket to communicate w/ Server Incorporate into a GUI PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

35. Video Feed Background Using MJPEG stream from Camera’s webserver Get the feed Put it in a component of the GUI PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

36. Motion JPEG Stream Stream looks like: Content-Type: image/jpeg Content-Length: -- PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

37. Making it usable Parse each JPEG out of the stream Update GUI when JPEG ready Continuously Paint image to screen PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

38. Video Feed Implementation StreamParser splits up Axis’ MJPEG stream into JPEGs strips out headers and looks for start of jpeg happens to be: 0xFF 0xD8 constantly gets byte arrays (segments) when –boundary found, jpeg is ready AxisCamera extends JComponent can be added directly to GUI Listener of StreamParser Runs as a Thread from GUI calling StreamParser PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

39. GUI Cockpit is the Client, connects directly to Server starts a Thread of AxisCamera to get images adds an AxisCamera to it’s JFrame puts these 2 connections together to operate as 1 Using awt and swing has the actual controls PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

40. Relations PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

41. Reuse between different IP cameras Control Side Server only need to change (if necessary) output toggle command (talk about this later) Video Feed Side Since MJPEG Streams are not standardized vary from camera to camera StreamParser and “AxisCamera” should work for any camera giving a MJPEG Stream as long as you know the boundary PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

42. 1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  CameraCamera  MicroprocessorMicroprocessor  Car Implementation Animated Diagram ObjectivesRequirementsDesign & ImplementationFuture Development

43. Using a microprocessor Initial coding/design on Dragon12 development board (MC9S12DP256) has a bus speed of 24MHz has registers for a Pulse Width Modulator and an Enhanced Capture Timer Transferred to DragonFly12 (on the car) Bus speed 48MHz Fewer ports PC  CameraCamera  MicroprocessorMicroprocessor  Car Design: Camera to Microprocessor ObjectivesRequirementsDesign & ImplementationFuture Development

44. Design: Camera to Microprocessor Camera’s alarm output can be activated about every 10 ms Initially 10-120 ms speed and 130-240 ms steering in increments of 10 ms (alternating) Problems encountered: Camera couldn’t be activated this fast when using the camera’s program for activating the output Slowed to around 0.1 second when streaming video Very inconsistent signal ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

45. Design: Camera to Microprocessor Changed to four bit signal in the form XXXX Still takes at least 250 ms to send signal Solved using Open Office word processor, Google, and beagle SpeedSteering ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

46. Design: Camera to Microprocessor Solution: Wrote program to test input/output files (GPIOx) to find correct output file Using ioctl system call, output can be activated in less than 1 ms Number of signals sent (0-14) determine speed and direction of car Longer signal sent to mark end of signal sequence ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

47. # pulsesSteeringSpeed 000 10 210 301 4 1 511 602 7 2 812 903 103 1113 1204 134 1414 PC  CameraCamera  MicroprocessorMicroprocessor  Car

48. Design: Camera to Microprocessor Needed End of signal ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

49. Implementation ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

50. Implementation Amplifier ObjectivesRequirementsDesign & ImplementationFuture Development

51. 1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  CameraCamera  MicroprocessorMicroprocessor  Car Implementation Animated Diagram ObjectivesRequirementsDesign & ImplementationFuture Development

52. Implementation ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

53. Problems putting code onto the DragonFly12 Can’t download code from Codewarrior to DragonFly12 Has to be converted to format suitable for DragonFly12 Used EmbeddedGNU connects to D-Bug12 on Dragon12 Dragon12 connected to DragonFly12 PC  CameraCamera  MicroprocessorMicroprocessor  Car Design: Microprocessor to Car ObjectivesRequirementsDesign & ImplementationFuture Development

54. Design: Microprocessor to Car EmbeddedGNU ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

55. DragonFly12 Test Setup ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

56. Future Development What could we do if we had more time? Put everything on a separate car with reverse Apply backtracking Game Controller Wireless strength monitor ObjectivesRequirementsDesign & Implementation Future Development

57. Sales and Marketing You can place an order for yours TODAY! $700…. Next year’s capstone students? Cash and Visa No Checks w/o valid drivers license

58. Special Thanks George Hauser Ph.D., University of Rochester Tosh Kakar Ph.D., Washington State University Wayne Chu Frank Wornle Cory Stevens Make-a-wish foundation Little Caesars Delivery guys who deliver straight to Morken 212a Ourselves

59. Questions & Demo Video