NETWORK RC CAR © James Crosetto BS (Computer Science and Computer Engineering) Jeremy Ellison BS (Computer Engineering) Seth Schwiethale BS (Computer Science)

Member Bio’s James Crosetto Computer Science and Computer Engineering Pizza Guru Seth Schwiethale Computer Science Music Diversity Contributor Jeremy Ellison Computer Engineering Anti-Coldplay

Presentation Outline Objectives Goal Setting Innovate & Design Requirements Necessary Equipment Design and Implementation Code Review Future Development Where to go from here ObjectivesRequirementsDesign & ImplementationFuture Development

Objectives Goal Setting Project Overview Functional Objectives Learning Objectives Innovative development Radio Frequency vs. Internet Computer vs. IP Camera Project OverviewFunctional ObjectivesLearning ObjectivesRF v. InternetAnimated Design ObjectivesRequirementsDesign & ImplementationFuture Development

Project Overview Expanding the range of a remote control car Control car over network Be able to have first person view of car’s location Build something cool Objectives Goal Setting ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional ObjectivesLearning ObjectivesRF v. InternetAnimated Design

Functional Objectives Establish connection between RC car and driver’s computer Get real time visual feed Control RC car User friendly GUI Objectives Goal Setting ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional Objectives Learning Objectives RF v. InternetAnimated Design

Learning Objectives Understand Wireless Communication Efficient methods of sending and receiving data Embedded Systems Microprocessors Assembly Language Linux / C R/C car design and functionality Servos Pulse Width Modulation Objectives Goal Setting ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional Objectives Learning Objectives RF v. InternetAnimated Design

Radio Frequency ft Line-of-sight Object interference Internet Network Advanced Range Non-line-of-sight Objectives Innovative Development ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional Objectives Learning Objectives RF v. InternetAnimated Design

Objectives Animated Design Progression ObjectivesRequirementsDesign & ImplementationFuture Development Project OverviewFunctional Objectives Learning Objectives RF v. InternetAnimated Design

Requirements Necessary Components Car Camera Microprocessor User’s PC ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

Requirements The Car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

3 Leads: Ground V cc Pulse width modulation Requirements The Car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

Requirements The Car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

Square Pulse wave of ms repeats every ~20ms Width of pulse determines the position of the servo with 1.5ms as the normal center The amplitude of the pulse is from the reference level to the V cc V cc = V Requirements The Car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

Power: 5.1 V DC, max 3.5 W Alarm output (motion, audio, external) Open API for software integration CPU, video processing and compression; Ram: 32MB Flash: 8MB Requirements IP Camera ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

Receives and translates signals from the transistor output of the IP camera Sends translated signals to steering box and speed control Programmed with C and Assembly Requirements Microprocessor DragonFly12 ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

Requirements product research – end user View video stream from camera Send commands to the car ObjectivesRequirementsDesign & ImplementationFuture Development CarCameraMicroprocessorUser’s PC

Unexpected outcome of design… Swarmed with applications to be our advisor:

Project Costs: Camera One time Purchase: $275 Little Caesars Hot-N-Ready: $600 annual cost Traveling Costs Seminar in Hawaii - $800/member Still pending department approval…

Implementation & Design ObjectivesRequirementsDesign & ImplementationFuture Development

PARTYING? How to accomplish our goals?

NO!!!

STUDYING?

YES!

Implementation 3 Major Lines of Communication PC  Camera Camera  Microprocessor Microprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  CameraCamera  MicroprocessorMicroprocessor  Car Implementation Animated Diagram ObjectivesRequirementsDesign & ImplementationFuture Development

Design: PC to Camera Software requirements to communicate Server to take commands (camera side) Client to give commands (user pc side) GUI to show video feed Reusability PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Server Background ARTPEC processor running Linux on a armv4tl architecture Server written in C Compiled using an x86 to ARM cross- compiler PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Crashing Ubuntu v6.06 LTSv6.10 V8.10v8.04 v7.10v7.04 ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Server Implementation TCP/IP Socket can be thought of as a file Basic Process create socket bind to port listen on port for connections accept connection handle input PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Server code ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Server implementation ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Server implementation ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Client Background Java for cross-platform TCP/IP Socket to communicate w/ Server Incorporate into a GUI PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Video Feed Background Using MJPEG stream from Camera’s webserver Get the feed Put it in a component of the GUI PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Motion JPEG Stream Stream looks like: Content-Type: image/jpeg Content-Length: -- PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Making it usable Parse each JPEG out of the stream Update GUI when JPEG ready Continuously Paint image to screen PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Video Feed Implementation StreamParser splits up Axis’ MJPEG stream into JPEGs strips out headers and looks for start of jpeg happens to be: 0xFF 0xD8 constantly gets byte arrays (segments) when –boundary found, jpeg is ready AxisCamera extends JComponent can be added directly to GUI Listener of StreamParser Runs as a Thread from GUI calling StreamParser PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

GUI Cockpit is the Client, connects directly to Server starts a Thread of AxisCamera to get images adds an AxisCamera to it’s JFrame puts these 2 connections together to operate as 1 Using awt and swing has the actual controls PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Relations PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

Reuse between different IP cameras Control Side Server only need to change (if necessary) output toggle command (talk about this later) Video Feed Side Since MJPEG Streams are not standardized vary from camera to camera StreamParser and “AxisCamera” should work for any camera giving a MJPEG Stream as long as you know the boundary PC  CameraCamera  MicroprocessorMicroprocessor  Car ObjectivesRequirementsDesign & ImplementationFuture Development

1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  CameraCamera  MicroprocessorMicroprocessor  Car Implementation Animated Diagram ObjectivesRequirementsDesign & ImplementationFuture Development

Using a microprocessor Initial coding/design on Dragon12 development board (MC9S12DP256) has a bus speed of 24MHz has registers for a Pulse Width Modulator and an Enhanced Capture Timer Transferred to DragonFly12 (on the car) Bus speed 48MHz Fewer ports PC  CameraCamera  MicroprocessorMicroprocessor  Car Design: Camera to Microprocessor ObjectivesRequirementsDesign & ImplementationFuture Development

Design: Camera to Microprocessor Camera’s alarm output can be activated about every 10 ms Initially ms speed and ms steering in increments of 10 ms (alternating) Problems encountered: Camera couldn’t be activated this fast when using the camera’s program for activating the output Slowed to around 0.1 second when streaming video Very inconsistent signal ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Design: Camera to Microprocessor Changed to four bit signal in the form XXXX Still takes at least 250 ms to send signal Solved using Open Office word processor, Google, and beagle SpeedSteering ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Design: Camera to Microprocessor Solution: Wrote program to test input/output files (GPIOx) to find correct output file Using ioctl system call, output can be activated in less than 1 ms Number of signals sent (0-14) determine speed and direction of car Longer signal sent to mark end of signal sequence ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

# pulsesSteeringSpeed PC  CameraCamera  MicroprocessorMicroprocessor  Car

Design: Camera to Microprocessor Needed End of signal ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Implementation ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Implementation Amplifier ObjectivesRequirementsDesign & ImplementationFuture Development

1. PC to Camera 2. Camera to Microprocessor 3. Microprocessor to Car PC  CameraCamera  MicroprocessorMicroprocessor  Car Implementation Animated Diagram ObjectivesRequirementsDesign & ImplementationFuture Development

Implementation ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Problems putting code onto the DragonFly12 Can’t download code from Codewarrior to DragonFly12 Has to be converted to format suitable for DragonFly12 Used EmbeddedGNU connects to D-Bug12 on Dragon12 Dragon12 connected to DragonFly12 PC  CameraCamera  MicroprocessorMicroprocessor  Car Design: Microprocessor to Car ObjectivesRequirementsDesign & ImplementationFuture Development

Design: Microprocessor to Car EmbeddedGNU ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

DragonFly12 Test Setup ObjectivesRequirementsDesign & ImplementationFuture Development PC  CameraCamera  MicroprocessorMicroprocessor  Car

Future Development What could we do if we had more time? Put everything on a separate car with reverse Apply backtracking Game Controller Wireless strength monitor ObjectivesRequirementsDesign & Implementation Future Development

Sales and Marketing You can place an order for yours TODAY! $700…. Next year’s capstone students? Cash and Visa No Checks w/o valid drivers license

Special Thanks George Hauser Ph.D., University of Rochester Tosh Kakar Ph.D., Washington State University Wayne Chu Frank Wornle Cory Stevens Make-a-wish foundation Little Caesars Delivery guys who deliver straight to Morken 212a Ourselves

Questions & Demo Video