1 Integrated Project (EIE330) Dr Daniel Lun Department of Electronic and Information Engineering Tel: Office: DE637 URL: THE HONG KONG POLYTECHNIC UNIVERSITY Department of Electronic and Information Engineering

2 Objectives Apply knowledge acquired in other subjects in an integrated manner Provide students with the experience of project development Technical challenges Software development System integration System troubleshooting Non-technical challenges Teamwork, time management, self-learning and critical thinking, problem solving, communications, etc. Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

3 Topic this year Robot Car Race – Augmented Reality To enable first person experience in a real car race Not just a computer game, real robot cars are racing and controlled by you Control a robot car as if sitting inside the car Augmented reality (AR) is a field of computer research which deals with the combination of real-world and computer- generated data Different from Virtual reality (VR) – simulate the real world by computer-generated data In this project, live video will be combined with 3D graphics in a robot car race Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

4 Things to do Learn the features, working principle and implementation methods of different sub-systems required in this project A series of lab exercises will guide you to achieve the minimum of the project requirement Find a partner to form a group Compete in the qualifying round of the robot car racing Summarize the experience and devise better solutions to improve the performance of your car You are asked to prepare a progress report to indicate your plan Improve your programs and compete in the final round of the robot car racing You are also required to make a presentation and a final report (Web page) to elaborate your improvement work and the result Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

5 Rules of the Game Each team is provided with a LEGO Mindstorms Robot Car Each team is provided with a steering wheel installed with a motion tracking device Develop the system that uses the motion tracking device to control the LEGO Mindstorms Robot Car to run on a racing track The car that uses the least amount of time wins the game Bonus marks will be given to the teams that develop good graphics Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

6 The Final Racing Track (subject to minor changes) Start Finish Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

7 Architecture of the Car Racing System Motion Tracking Server Game Server (Mindstorms Control ) Motion Tracking System Network Video Server Marker 3D Graphics Server Video + 3D Graphics Bluetooth radio (Control car movement) RF (Transmit video) Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun Your program

8 Things to learn To obtain the position information of the markers through the Motion Tracking Server using the Windows Socket technology To process the data obtained from the Motion Tracking Server for controlling the Mindstorms Robot Car To display the video received from the wireless camera in Ogre using the DirectShow technology To detect objects in the video using simply image processing algorithms Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

9 Resource Interactive Multimedia Technology Lab Polhemus Liberty Latus wireless motion tracking system LEGO Mindstorms robot cars Panoramic projector system High performance computer systems with 24” HD monitors Ogre 3D game engine, 3D Studio Max, Visual Studio.NET Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

10 Interactive Multimedia Lab (DE505a) To be used exclusively by BScIMT students doing EIE330

11 Polhemus Liberty Latus The LIBERTY LATUS (Large Area Tracking Untethered System) is a magnetic based motion tracking system Design to track the position and orientation (P&O) of multiple objects in the 3D space Comprise 3 major components: SEU – System Electronic Unit Receptors Wireless Markers Markers’ P&O will be detected by the Receptors and processed by the SEU Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

12 Steering Wheel with Latus A Latus Marker is installed on a steering wheel Continuously send out its 6-D information to the Motion Tracking Server All controls of car movement are made based on the 6-D info of the marker LATUS marker Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

13 LEGO Mindstorms® Robots The LEGO Mindstorms® is a robotic building system consisting of: The NXT Intelligent Brick: the brain of the system Sensors and servo motors LEGO TECHNIC Elements Programming software

14 MINDSTORMS Robot Car NXT Wireless Video Camera BlueTooth Wireless Motion Control

15 The OGRE 3D Graphics Engine OGRE stands for Object-Oriented Graphics Rendering Engine OO interface designed to minimize the effort required to render 3D scenes Independent of 3D implementation e.g. Direct3D, OpenGL, Glide etc. Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun Contain example frameworks Common requirements for 3D rendering are done for the user automatically

16 Windows DirectShow® DirectShow® is a middleware architecture that provides a pipeline for media playback and capture DirectShow uses filters to manage and manipulate multimedia data Filters perform actions, such as parsing, decoding, formatting, or rendering, on a multimedia streams One can create their own filters for video processing Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

17 Schedule and Arrangement Lectures (CF504) Jan 15, Jan 18, Feb 1 (Gp 1), Feb 12 (Gp 2), Feb 22 (Gp 1), Feb 26 (Gp 2), Mar 11 and Apr 1 Lab and tutorial (CF504) All other timeslots on Tuesday and Friday Note the labs on Feb 15 and 29 are only for group 1, labs on Feb 19 and Mar 4 are only group 2 Submissions Lab report and demonstrations x 5 (Note the submission deadlines) Progress report – Apr 8 Final report – May 2 Logbook – check and sign by assessor on Apr 8 and submit with the final report on May 2 Qualifying Round racing – Mar 28, Final Knockout – Apr 25 Presentation – May 2 Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

18 Assessment Weighting Assessment TypeWeighting Lab exercises20% Logbook / Progress report / Final report / Presentation 20% Result in Qualifying Round20% Ranking in Final30% Project Management10% Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

19 Assessment Criteria Lab exercises Finish all guided and open-ended exercises as listed in the lab sheets Altogether 5 laboratory exercises For every lab, each student needs to submit A lab report (the computer program of the open-ended exercise) and make a demonstration of his/her work at the specific time Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

20 Assessment Criteria Logbook Individually keep a logbook (a diary) on the lab and project work performed Check by the assessor on Apr 8 Submit the logbook on May 2 Can be a handwritten one or a computer file Assessment will be made based on the completeness of the contents for recording the findings and results of the project Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

21 Assessment Criteria Progress report Submit a progress report on or before Apr 8 Should include the following: The topic and objective of the game The functional and technical specifications Preliminary results obtained A summary of the experience gained in the project and propose improvement plan The general criteria for assessing a progress report include: Completeness of the above items Feasibility of the improvement plan Presentation of the report Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

22 Assessment Criteria Final report and Presentation Submit a final report in the form of a Web page on or before May 2 Based on the Web page, make a presentation (in English) The contents of the Web page and presentation should include: Video clip(s) that describes the operation of the game developed Actual techniques adopted to implement each sub-system of the game Results achieved by using those techniques The ways to manage the resources in the project Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

23 Assessment Criteria Qualifying Round All team are required to attend the Qualifying Round of the Robot Car Race on Mar 28 Each team member is required to control a Robot Car to finish a round of racing track The average time required by the two team members will be counted and graded Important: Teams that cannot finish within the minimum required time will not be allowed to join the Final Knockout Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

24 Assessment Criteria Final Knockout Qualified teams are invited to join the Final Knockout 5 knockout rounds Teams with slower result in the Qualifying Round need to play in Round 1, the others will start in Round 2 Each set contains two games played by each team member The team with the faster average time proceeds to the next round Marks distribution: Champion: 22Finalist:18 Semi-finalist:14Quarter-finalist:10 Round 2: 6Round 1: 2 A bonus 8 marks will be given to the effects and graphics added to the game Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

25 Assessment Criteria Project Management A total of 10 marks are allocated to this item To encourage the teams using good approaches for managing the resources of the project: Manpower Time Lab facilities Need to specify the approach in resource management in the final report and presentation Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

26 Why project management? A project involves the interaction of different parties A project will be successful only if all parties do their job right By having good project management, we can Achieve the project goal on time and within the budget Keep customers (e.g., Professors) happy Keep the team focus on the goal and work well Everyone shares the load Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

27 Scope, Resources, Schedule & Customers Project Management Project Planning and Implementation Define objectives / requirements Define work/tasks Identify resources Plan schedule Iterate Plan modification Execution Management Get objective signed-off Track plan progress Communicate within team Customer communications Secure resources Project discipline TechnicalPeople Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

28 Project Planning The process of project development is full of uncertainties The more uncertainties you have, the less possible that the project can be successful Uncertainties can be reduced by having a good feasibility study before the project Background research Understand the resource required and acquired Risk analysis The feasibility study has been prepared for you in the lab exercises Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

29 Requirement Analysis The purpose is to identify and document the exact requirements for the project Much interaction must take place between the customer (e.g. Professor) and the developer (e.g. students) Should be well documented since very often it serves as an agreement of the functions that will be provided by the system developed The document is sometimes called user specifications or functional specifications May form the basis for project commissioning Your progress report should contain the above items Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

30 System design and specification Split into two sub-phases: architectural and detailed design Architectural design Define the overall organization of the system in terms of high- level components and their interaction Detailed design Lower level modules in each component and their interfaces Example: In software development Architectural design – the function and relationship of classes Detailed design – the internal modules and interfaces of each class Should be well documented The document is sometimes called technical specifications Your progress report should contain the above items Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

31 Implementation – Work breakdown Split project into tasks and estimate time and resources required to complete each task Organize tasks concurrently to make optimal use of workforce Minimize task dependencies to avoid delays caused by one task waiting for another to complete Should be detailed in the progress and final report Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

32 Example – Activity Timeline and Staff Allocation 1/6/061/71/81/91/101/111/121/1/07 Lun Hsung Cheung EO T1: Define objectives and requirements T2: Room renovation T3: Hiring staff T4: Purchase equipment T5: Installation of equipment M1: Milestone 1 - Demo room equipment T6: Testing motion tracking system T7: Testing game engine T8: Testing other hardware and software T9: Prepare preliminary demo M2: Milestone 2 - Preliminary demo T10: Integrating all sub-systems T11: Develop final demo program T12: Prepare documents M3: Milestone 3 - Final demo T2 T3 T4 T5 M1 T6 T7 T8 T9 M2 T3 T10T11 T12 M3 T1

33 Resource Management About how to manage the manpower, time and facilities required in the project Some bad practices of students: “We didn’t allocate task to members since we work together all the time.” A very bad practice. Will get very low marks in project management “There’s not a leader in our team. We always make decision together.” For small team, the role of leader may not be very obvious. But I would still suggest you to find a leader in the team so as to practice the way of managing large projects. Besides, it can also give you convenience particularly if there is dispute among members. Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

34 Resource Management “My team member didn’t do anything. I decided to leave him alone. The work submitted was done by me.” See, finding suitable manpower is very important. That team member certainly will fail, since team members will be individually assessed. The marks of the group in project management will also be deducted since the project leader failed to estimate the risk at the planning stage and make appropriate contingency measure. “The demo is tomorrow. I have finished everything at home already today. I just need to change the IP address in the program then it will run in the lab.” Last year many students tried this but found that at the end the program simply could not compile and run. Remember, unless you have seen your program ever run correctly in the lab, never assume any result you get at home can be repeated in the lab. Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

35 Resource Management “It is very troublesome to book the Interactive Multimedia Lab. We plan to do the project at home.” Last year many students failed to deliver their project just because they did not spend enough time in the lab. There are many sub-systems in the project that their realization can only be done in the lab (DE505a). This year we will count the time that the teams spent in the lab as one of the assessment criteria for project management. “Our team was very busy in many different things so we skipped all the lab works. But we managed to use the last week to finish everything and made a good demo of our robot.” Assume that you are doing this project for your customer. If I were the customer of your project, I would never hire your team to do project again! All customers want to see both progress and result. So your marks in project management will be deducted. Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun

36 Some Useful Advices The lab exercises serve as the background studies to prepare you to finish the project Not an extra work but a part of your project Remember to finish them all by yourselves Don’t wait until the last week to do your project Software will not work if you develop it in a rush Many teams failed to do the demo last year just because of that This is not a computer science project but a computer engineering project Is not implemented with your computer only Involve many sub-systems that may not work as you desired if you don’t really try them in the lab Department of ELECTRONIC AND INFORMATION ENGINEERING Introduction by Dr Daniel Lun