Presentation is loading. Please wait.

Presentation is loading. Please wait.

May 19, 2005 Presentation General overview - Barry Kurtz

Similar presentations


Presentation on theme: "May 19, 2005 Presentation General overview - Barry Kurtz"— Presentation transcript:

1 May 19, 2005 Presentation General overview - Barry Kurtz
Set up of grid – Barry Wilkinson Distributed MATLAB – Rahman Tashakkori Course overview – Barry Kurtz Cryptography course - Shan Suthaharan Image processing course - Rahman Tashakkori Future of grid computing - Barry Wilkinson Intelligent decision making course – Dave Powell Remaining courses – Barry Kurtz Future Plans – Barry Kurtz

2 Overview Barry Kurtz

3 A Consortium to Promote Computational Science and High Performance Computing
Lead Institution: Appalachian State Partnering Schools: Elon University, High Point University, Lenoir-Rhyne College, North Carolina A & T, UNC Charlotte, UNC Greensboro, UNC Pembroke, Western Carolina University Funding: $650,000 Duration: July 1, 2004 – June 30, 2006

4

5 Mission Our mission is to provide the opportunity for undergraduate students at comprehensive universities to study computational science and high performance computing at a level comparable to students at Research I institutions, to promote faculty research by involving undergraduate students in cutting-edge research projects, and to stimulate economic development by promoting grid computing methodologies throughout North Carolina.

6 Vision Our vision is that by pooling knowledge, resources, and courses at eight collaborating comprehensive universities, by establishing a shared grid computing network, and by exporting grid technology to local IT companies, we can satisfy our mission statement.

7 Goal Our goal is to provide students at comprehensive universities the opportunity to take advanced courses in computational science and high performance computing at the undergraduate level and allow these students to participate in cutting-edge team-oriented research projects in support of faculty research programs.

8 Faculty and Institutions
Barry L. Kurtz and Rahman Tashakkori Appalachian State David Powell and Joel Hollingsworth Elon University Bill Hightower and Roger Shore High Point University* Dick Hull and Bjarne Berg Lenoir-Rhyne College* Yaohang Li and Stephen Providence North Carolina A & T Barry Wilkinson UNC Charlotte* Shan Suthaharan and Sue Lea UNC Greensboro Doek-Hyun Hwang and William Campbell UNC Pembroke Barry Wilkinson and Mark Holliday Western Carolina * Joins the consortium in year two

9 Hardware Acquisitions
A Computer Cluster Eight Dell Precision 360 or 370 workstations High Speed Switch Firewall There are currently eight clusters purchased; a ninth cluster at UNC Charlotte will be added soon

10 Original Equipment Projections
Year One $15,000 per cluster budgeted for six schools $10,000 for partial cluster at two schools Total allocation: $110,000 Year Two $5,000 to complete cluster at two schools Total allocation: $10,000 Actual Equipment Costs Six months after the projected costs were estimated, the actual equipment (identical to the promised equipment) only cost $12,000 per cluster

11 Actual Equipment Expenditures
Year One $12,000 per cluster for eight schools (the two partial clusters are made complete) An additional cluster for Barry Wilkinson who is returning to UNC Charlotte (our ninth partner) $2,000 is still left in reserve Year Two $10,000 to complete the partial clusters is available Combined with the $2000 in reserve we can fund a tenth cluster for another UNC school that is willing to join our consortium

12 Software Grid Computing Software MATLAB MPI Globus Condor
Toolboxes: Wavelets, Signal Processing, Image Processing, Neural Networks, Statistics, Bioinformatics Distributed MatLab MPI Standard MPI Grid-enabled MPI

13 Set up of grid Barry Wilkinson

14 Grid Support Goals Create a working grid with clusters at every university in the consortium. Use that grid in courses that are jointly taught by faculty from several of the universities. Participating Sites

15 Current Status Summer 2004: Initial installation of the NMI Distribution including Globus Toolkit 3.2 and Condor-G Summer 2004: Developed projects testing the distribution (web services, grid services, GRAM, Condor-G) Summer 2004: Distribution CD developed Summer 2004: First load testing in summer workshop at ASU Participating Sites

16 Current Status Fall 2004: Clusters acquired and installed at all sites
Fall 2004: All clusters operational with basic functionality: Linux operating system and cluster software Participating Sites

17 Current Status Fall 2004: Intense testing of the grid software in the Grid Computing course taught by Barry Wilkinson Fall 2004: Extended the software being used to include a MPI project developed by Barry Wilkinson and a grid workflow editor, GridNexus, developed at UNC-Wilmington Participating Sites

18 Current Status Spring 2005: Installation of a newer NMI Distribution (still GT3.2) at Elon and WCU Spring 2005:Testing of the new version of the software in the Intelligent Decision Making course at Elon and WCU Spring 2005: Joel Hollingsworth developed a new grid services project Participating Sites

19 Future Plans Summer 2005: Installation of a newer NMI Distribution (changing to GT4!) Summer 2005: Redo all the projects using GT4; develop more projects Summer 2005: Retest everything in the summer workshop Summer 2005: Create a certificate authority at Elon to allow multi-site operation Participating Sites

20 Future Plans Fall 2005: Test the new installations and project versions in Barry Wilkinson’s Grid Computing course Fall 2005: Further test the grid software in the capstone course and the other courses Spring 2006: Revise the projects (and perhaps reinstall) based on the Fall experience Spring 2006:Use in the spring courses Participating Sites

21 Summary It has been a learning experience for everyone involved.
It has been a successful experience with students getting working experience with grid software. Teamwork and inter-university collaboration have been the keys to our success. Participating Sites

22 Summary Much remains to be done transition to GT4 project revisions
additional projects certificate authority and multi-site grid operation Participating Sites

23 Distributed MATLAB Rahman Tashakkori

24 Distributed Toolbox The MathWorks web site was the main source of the MATLAB related information on this presentation The Distributed Computing Toolbox works with the MATLAB Distributed Computing Engine to execute coarse-grained MATLAB algorithms and Simulink models in a cluster of computers It allows prototyping and developing applications in the MATLAB environment and then use the Distributed Computing Toolbox to divide them into independent tasks. The MATLAB Distributed Computing Engine evaluates these tasks on remote MATLAB sessions.

25 Distributed Toolbox: Key Features
Distributed execution of coarse-grained MATLAB algorithms and Simulink models on remote MATLAB sessions Control of the distributed computing process via a function-based or an object-based interface Distributed processing on both homogeneous and heterogeneous platforms Support for synchronous and asynchronous operations Access to single or multiple clusters by single or multiple users

26 Distributed Toolbox

27 Creating and Submitting Jobs with the Distributed Computing Toolbox
The toolbox includes functions for defining jobs, dividing them into tasks, sending them to the MATLAB Distributed Computing Engine for execution, and retrieving the results. The complete process includes five steps: Finding a job manager Creating a job Creating tasks Submitting the job to the job queue Retrieving results

28 Problems Network has to be able to do multicast
It is not X-term friendly, best to run the jobs from console All machines should have the same version of MATLAB It requires all Unix-based machines (manager and workers) to run the same version of kernel In case that a firewall is used, specific ports should be left open to establish workers-server and server-license manager connectivity

29 Course Overview Barry Kurtz

30 Courses Cryptography and Network Security Fall 04-05 Shan Suthaharan
Barry L. Kurtz Digital Image Processing Rahman Tashakkori Sue Lea Grid Computing Barry Wilkinson UNCW faculty Intelligent Decision Making Spring 05-06 Dave Powell, Joel Hollingsworth, Mark Holliday Monte Carlo Methods Yaohang Li Parallel Algorithms Steven Providence William Campbell Bioinformatics Summer Doek-Hyun Hwang Capstone Research Course Fall 05 Mark Holliday, et. al.

31 Research Outcomes Publications Presentations Projects
2 in print 2 under review Presentations 7 completed Projects 13 completed 2 planned for summer Follow Up Grant Proposals 3 planned for the National Science Foundation 1 planned for another agency

32 Cryptography and Network Security Shan Suthaharan

33 Cryptography and Network Security
Host Institution – UNC Greensboro Dr. Shan Suthaharan, primary instructor Fall 2004, 15 students at UNCG Remote Classrooms Dr. Barry Kurtz, AppState, team instructor 2 students at AppState

34 Elliptic Curve Cryptography
Stephanie Rednour worked under the direction of B. Kurtz on this special research project

35 GUI for RC5 Encryption Ramu Pulipati worked under the direction of Shan Suthaharan on this special research project.

36 Digital Image Processing Rahman Tashakkori

37 Digital Image Processing
Host Institution – ASU Dr. Rahman Tashakkori Fall 2004, 20 students at ASU Remote Classrooms Dr. Sue Lea, UNCG 2 students at UNCG

38 Major Topics Introduction Mathematical Background Fundamentals
Intensity Transformations and Spatial Filtering Frequency Domain Processing Image Restoration Image Compression Wavelet Analysis

39 Image Processing Projects, ASU
These projects were completed under the direct supervision of Dr. Tashakkori Investigating content-based search techniques in medical images of different modalities – Steve Heffner This project lead to Steve’s honor’s thesis An Efficient Medical Image Content-based Search Approach, May 2005 Parallel Implementation of lifting schemes – Tim Racz This project lead to Tim’s M.S. thesis Parallel implementation of Hexagonal Lifting Schemes, In process

40 Image Processing Projects, ASU
Other Projects: A Java-based enhancement Toolbox. This toolbox includes several different modules that are developed by 8 students throughput the semester. A C++-based enhancement Toolbox. This toolbox includes several different modules that are developed by 5 students. Histogram equalization implementation of color images A morphing and edge detection Toolbox. This toolbox includes three different modules that are developed by 3 of our students throughout the semester An image cropping Toolbox. A high- and low-pass filtering Toolbox

41 Image Processing Project, UNCG
David Waizenegger and Sarah Parker worked on this project under Dr. Lea’s supervision The students investigated time sequences of (ocean) wave crest images. The wave crest data is not clean data; crests may be erroneously connected to form contours. Students were asked to come up with an automatic technique for determining the angle through which the image needed to be rotated in order to make the crests approximately parallel to the bottom of the rotated image. Accomplishing such a rotation simplifies analysis of the sequence of images to infer the bottom depth near the shore.

42 Future Plans – Fall 2005 Continue teaching the course in the consortium Expand and improve existing Toolboxes Create new Toolboxes Establish a team-based research problems to includes collaboration across campuses Utilize the grid to solve some of the image processing problems

43 Student Presentations
Steve Heffner, Investigating content-based search techniques in medical images of different modalities, The 8th Annual Celebration of Student Research and Creative Endeavors, April 19, Appalachian State University

44 The Future of Grid Computing Barry Wilkinson

45 Intelligent Decision Making Dave Powell

46 Intelligent Decision Making
Elon Instructors: Powell, Hollingsworth WCU Instructor: Holliday Using Inexpensive High Performance Computing to design products faster, better and cheaper.

47 Overview Motivation: Need for High Performance Computing for Global Competitiveness Course Enrollment, Topics and Projects Ideal Capstone Course Key Benefits Key Pedagogies for NCREN Key Challenges Student Survey Suggestions Research Outcomes Future Work

48

49

50 Optimization Customers
Airbus AlliedSignal Black & Decker Boeing Bombardier Bridgestone/Firestone Corning Daimler Delphi Packard Ford General Electric General Motors Hitachi Computer Kodak Lockheed-Martn Mitsubishi Motorola NASA Nippon Sheet Glass Northrup Grumman Otis Elevators PPG Pratt & Whitney Raytheon Rolls Royce Sony Toshiba TRW U.S. Army, Navy, & Air Force Voith Xerox York

51

52

53 Key Questions For Engineer and Student
How do I pose the optimization problem? (theory) What type optimization algorithms are available? (theory) How do I pick the correct algorithm? (theory) What are some of the available optimization codes? (public domain) How can I quickly apply code to my “customers” problem? (wrapping) How can I use grid computing to quickly arrive at a solution?

54 Course Details Host Institution – Elon University Student Breakdown
Team taught: Hollingsworth and Powell, Elon University, Holliday, Western Carolina Student Breakdown Eight students at Elon Three students at Western Carolina Five students at Appalachian State Average GPA:

55 Major Topics A Mathematical Programming Language (AMPL)
Object wrapping with adapter of facade design pattern Condor-G for multiple and parallel job submission Grid Services (Globus 3.2) Introduction to Formulation and Classification of Optimization Models Elements of Improving Search Based Optimization Algorithms Formulation and Classification of Linear Problems Simplex algorithm Sensitivity analysis Formulation of Unconstrained NLP Golden Section and Gradient Search Formulation of Constrained NLP Penalty Methods Formulation of Mixed Integer Problems

56 Programming Projects Used the Façade Design Pattern to wrap the AMPL commercial code and called it from a customized Java Swing GUI. AMPL Java Wrapper AMPL Gui Application Specific Interface APIs User Library

57 Programming Projects Coupled a third party nonlinear constrained optimizer AMPL Java Wrapper AMPL Gui Application Specific Interface APIs User Library APIs Student Optimizer

58 Programming Projects Develop optimization grid service and validate/demonstrate with two separate client GUIs. Grid at Elon AMPL Java Wrapper Gui Application Specific Interface User Library AMPL Java Wrapper AMPL Client Machine at ASU Student Optimizer

59 ASU Submission

60 Programming Projects Developed Condor-G scripts to submit and execute optimization from multiple starting points on Elon 8 node grid running Globus 3.2.

61 Ideal Capstone Course Languages/Formats AMPL C Fortran Java Soap XML
WSDL WSDD Concepts Client/Server Wrappers JNI Certificates Pairwise Programming SOA Proxy Reuse Applied Math Finite Differences Taylor Series Lagrange Multipliers Simplex Penalty Functions Convex/Concave Courses Calculus 1 and 2 Linear Algebra Programming 1, 2, 3 Programming Languages Environments/Tools Linux Apache/Tomcat/Axis Web Services Grid Services Condor Ant Abstractions Optimization Formulations

62 Key Benefits Students Faculty/University
Course that would not otherwise be taught. Expertise of faculty. A grid node with 8 machines. Push outside comfort zone. Comparison with peers. State of the art exposure. Faculty/University New Computing Equipment New NCREN Equipment Dialog with colleagues New skills

63 Key Pedagogies for NCREN
Pairwise Programming Blackboard Digital Dropbox Discussion Board Grades Assignment Posting Captured and posted class for video streaming Semester project Define, Specify, Design, Impl, Test Netmeeting Skype Instant Messenger HW Solution Posted immediately

64 Key Challenges Different start and end times
Different number of credit hours. Different spring breaks. Different off days (e.g. Easter) System administration.

65 Student Surveys More frequent switching of project teams.
More individual work. Course was too much work. Felt like two courses in one. Better course description to emphasize amount of Math involved. More interaction/presentations between the schools to hear more from the other students. Same spring break for all campuses.

66 Research Outcomes Hollingsworth and Powell submitted paper, “Globus Grid Computing and AMPL: a Pragmatic Educational Environment for Real World, Engineering Design Optimization” to “The Fifth IASTED International Conference on Modeling, Simulation and Optimization”. Status: Accepted. Hollingsworth and Powell submitted paper, “Leveraging Grid Computing in an Intelligent Decision Making Course” to IASTED International Conference on Computers and Advanced Technology for Education Status: Accepted. Extended AMPL Commercial Tool Capable for being extended to do multiple objective optimization. (standalone and as grid service) Coupled multiple objective optimization package, NSGA2, with AMPL package. Coupled simulated annealing optimization package, ASA, with AMPL package.

67 Future Plans Install the new release of Globus Toolkit, GT4 and investigate assortment of third party parallel job submission tools from Platform Computing and Sun Microsystems. Extend NSGA2 AMPL coupling to evaluate populations in parallel. Investigate BPEL for workflow control of hierarchical optimization, simulation code evaluation, grid services and web services. Extend Eclipse Plug-ins to support grid computing

68 Other Courses Barry Kurtz

69 High Performance Computing and Monte Carlo Methods
Host Institution – North Carolina A&T State University Dr. Yaohang Li, Primary Instructor Spring 2005, 10 students enrolled at NCAT from Departments of Computer Science, Mechanical Engineering, Computational Science and Engineering, Physics, and Biology

70 Research Outcomes High Performance Computing and Monte Carlo Methods
Students completed term papers on the following topics Computational Financing Monte Carlo Computation on the Grid Ray Chasing Problem using Monte Carlo method Network Simulation Protein Structure Prediction using Markov Chain Monte Carlo Ab initio Monte Carlo

71 Parallel Algorithms Host Institution – North Carolina A & T
Dr. Stephen Providence, instructor Spring 2005, 5 students at NCA&T Remote Classrooms 2 students at Appalachian State supervised by Barry Kurtz 1 student at UNC Greensboro supervised by Shan Suthaharan

72 Outcomes Parallel Algorithms
Grid-enabled MPI was installed and used on the local computer clusters Students completed a sequence of assignments out of Barry Wilkinson’s textbook: Parallel Programming : Techniques and Applications Using Networked Workstations and Parallel Computers (2nd Edition)

73 Bioinformatics Computing
Host Institution – UNC Pembroke Dr. Deok-Hyun Hwang, primary instructor Summer I, June 2005 Remote Classrooms 2 students at Appalachian State University supervised by Barry Kurtz Enrollment at other institutions is not known at this time

74 Research Outcomes Bioinformatics Computing
Set up grids on Dell machines as well as Compaq machines Set up a stand alone Blast system with database Design and build globus-enabled HTC Blast on a grid Implement parallel MPICH-Blast on a grid

75 Capstone Research Course
Coordinator: Mark Holliday Goal: Provide an opportunity for a student to explore one of the course areas in further depth Offered: Fall 2005 Participating Sites

76 Capstone Research Course
Implementation: Student works with a faculty member in a specific course area Project presentation to the entire class using NCREN Mark Holliday provides grid support and overall coordination Participating Sites

77 The Future Barry Kurtz

78 What Remains to be Done Welcome three new schools into the consortium
UNC Charlotte, High Point University, Lenoir Rhyne College Invite additional schools to join the consortium Teach the Summer 2005 Workshop Intended for other UNC schools and IT industry At Appalachian State July 29-31, 2005 July 29: MatLab, MPI; July 30: Grid Setup; July 31: Grid Applications Teach the Capstone Research Course Hosted at Western Carolina by Mark Holliday Teach seven other courses for a second time Emphasis use of HPC hardware and software Involve student in cutting edge research projects

79 Future Plans Institutionalize the high performance computing courses at the host institutions Continue research work in high performance computing Sponsor student project work in high performance computing Apply for external funding from the NSF and other funding agencies


Download ppt "May 19, 2005 Presentation General overview - Barry Kurtz"

Similar presentations


Ads by Google