1. Parallel Programming Fall 2014 Tuesday/Thursday 2:00 pm - 3:15 pm Instructors
Dr. Barry Wilkinson
University of North Carolina Charlotte
Dr. Clayton Ferner
University of North Carolina Wilmington
© B. Wilkinson/Clayton Ferner. Modification date: Aug 21, 2014
outline.1 1 1

2. Outline
These slides constitute the course outline and include the course syllabus and course policies.

3. Parallel Computing
The use of multiple processors or computers to solve problems at a greater computational speed than using a single computer/processor.
Basic idea is simple – using N computers/ processors collectively on a problem should lead to a faster solution.
Can tackle problems that could not be solved in a reasonable time otherwise.
Also solve problems with higher precision and/or with more memory requirements.

4. Computer platforms for parallel computing
Multiple interconnected computers
- Cluster Computing, group of interconnected computers typically using Ethernet switch and physically all in one room.
2. A single computer system with multiple internal processors or cores
- Usually shares a common main memory
Computer system with attached graphic processing unit (GPU)
- GPUs have large number of execution cores, now used for high performance computing as well as original graphics application
Combination of above most likely.

5. UNCC cluster used in course
ssh to user gateway
cci-gridgw.uncc.edu
Login
coit-grid06
cci-grid07
cci-grid08
coit-grid01
coit-grid02
coit-grid03
coit-grid04
cci-grid09
C2050 GPU
(448 core)
C2050 GPU
(448 core)
K20 GPU
(2496 core)
cci-grid09:
Dual AMD Opteron GHz 16-core each, with 16 GB main memory
RAID1 disks,
redundant power supplies, VM, web server, user gateway login, SSH, NAT, DHCP, DNS, and LDAP, etc.
coit-grid01-4: Each two Xeon 3.4 Ghz processors, 8GB main memory
coit-grid06:
GPU server, with NVIDIA 2050 GPU
cci-grid08:
GPU server, E GHz 6-core Xeon processor with NVIDIA K20 GPU, 32 GB main memory
cci-grid07:
GPU server, X GHz quad-core Xeon processor with NVIDIA 2050 GPU, 12GB main memory
switch
cci-grid05: Four quad-core 2.93Ghz Xeon processors 64GB main memory
1.2 TB disk (RAID 5)
cci-grid05
All user’s home directories on cci-grid05 (NFS)
Has all three types of platform.
Note: Some names have been changed from coit- to cci-

6. UNCW cluster used in course

7. Your own computer
Generally you will need your own computer (desktop or laptop, Windows, Mac, or Linux), as some programming will be done on a local computer in addition to being able to access the remote clusters.
(You can use a lab computer although that might not be as convenient for you.)
All the software will be provided for download as a VirtualBox Ubuntu virtual machine. You will need to install VirtualBox. Detailed instructions will be provided.
Detailed instructions also be provided for a native installation on a Linux distribution if you wish to do that instead.

8. Course Prerequisites UNC-C Catalog prerequisites:
ITCS 2214 (Data Structures), and
ITCS 3181/3182 (Computer Architecture).
Computer Architecture not strictly necessary so long as you know what a computer consists of (processors, memory and I/O) and how it operates.
What is needed -- basic skills in C and Java.

9. Course Text
Parallel Programming: Techniques and Application Using Networked Workstations and Parallel Computers, 2nd edition, by B. Wilkinson and M. Allen, Prentice Hall Inc., 2005, ISBN
Not strictly necessary very detailed on-line notes provided, some derived from the course text – with updated materials since 2005

10. Home Page http://coit-grid01.uncc.edu/ITCS4145F14/ Password: rp19zb39
For course notes, assignments, announcements, etc.:
Password protected:
Username: pablo
Password: rp19zb39
Backup location: cs.uncc.edu/~abw/ITCS4145F14/

11. Course Contents
Parallel computers: architectural types, shared memory, message passing, interconnection networks, potential for increased speed.
Parallel patterns for structured parallel programming, motives, advantages, introduction to various patterns, tools.

12. Course Contents (Continued)
Message passing: MPI message passing APIs, send, receive, collective operations. Running MPI programs on a cluster, implement patterns with MPI.
More computation patterns and their application, divide and conquer, pipeline, iterative synchronous all-to-all, stencil.

13. Course Contents (Continued)
Programming with shared memory programming: Specifying parallelism, sharing data, critical sections, threads, OpenMP. Running threaded/ OpenMP programs on multi-core system, hybrid MPI/OpenMP programs.
CPU-GPU systems: data parallel pattern, GPU architecture, programming in CUDA, issues for achieving high performance.

14. Course Contents (Continued)
Algorithms and applications: Selection from:
Sorting algorithms
Searching algorithms
Numerical algorithms
Image processing algorithms

15. Course Grade Sites with +/- grading Sites without +/- grading
Undergraduate Grade Scale A A B B B C C C D D D F
Graduate
Grade Scale A A B B B C C C U
Undergraduate
Grade Scale A B C D F
Graduate
Grade Scale A B C U

16. Computation of Grades Class quizzes (2) 25% Assignments (5-7) 50%
Final exam 25% The assessment and percentages may be modified.
Read small print

17. Graduate students
There will be additional work in the assignments for graduate students, which will be extra credit for undergraduates.

18. Student Learning Outcomes (SLOs)
The learning outcomes for the UNC-W undergraduate parallel computing course are:
Students develop solutions and implement the solutions for scientific problems using multiple processors in a cluster.
Students acquire knowledge in writing programs which require processes to synchronize and exchange data using Message Passing Interface (API).
Students acquire knowledge in writing programs which require processes to spawn multiple treads that share memory on multiple core processors.
Students learn to profile parallel programs by placing event probes and to log them for analysis and tuning of the program performance.
The Estimate Curriculum Category Content (Semester hours) is:
Area
Core
Advanced
Algorithms
1.0
Software design
0.5
Data structures
0.25
Concepts of programming languages
Computer organization and architecture

19. Code of Student Academic Integrity
All submitted assignments must be your own work. Copied work or work done by more than one person (unless specifically instructed) will not be accepted - at the very minimum, zero credit - and may be subject to disciplinary action.
The Code of Student Academic Integrity applies to both assignments and quizzes/tests.
Required statement

20. Due dates for assignments
Will be posted on home page with assignment.
Assignments can be turned in late but with a penalty.
Late Penalty
< 24 hours 10%
> 24 hours and < 48 hours 20%
> 48 hours and < 72 hours 30%
> 72 hours 100%

21. Attendance Attendance is expected.
At UNC-C -- Attendance will be recorded. If you miss classes, it can have a deleterious effect on your grade:
After 4 missed classes without good reason:
2 marks off overall accumulated marks (out of 100) for each additional class missed.
If you must miss class, see the instructors beforehand to avoid losing marks.

22. Mid-Term Unsatisfactory Grades
UNC-Charlotte requires midterm unsatisfactory grades to be submitted by the end of the 8th week of class.
Mid term grades
Undergraduates: Satisfactory or D or F
Graduates: Satisfactory or C or U
will be based upon all work graded by week 7, which
probably will be two assignments and one class test.

23. Ask questions during class and participate in class discussion.
Expectations
Read all materials provided.
Read slides before class. Take additional notes during class
Ask questions during class and participate in class discussion.
outline.23 23

24. NCREN facility (North Carolina Research and Education Network)‏
Each site provides microphones for individuals to speak.
Please feel fee to speak at suitable times but remember NCREN needs recognize site and speaker, so first announce your institution and name.
Most microphones are turned on all the time. Any sound will be heard across all sites, so avoid unnecessary sounds.
Fans from a laptop can be especially disturbing, so move laptop away from a microphone. Turn cell phones off.
outline.24 24

25. NCREN Video Class Recording
The lectures are available on-line.
Details to be provided.
outline.25 25

26. Skype (will be on during office hours, username abwilkinson)
Instructors details
UNC-Charlotte Barry Wilkinson Department of Computer Science University of North Carolina, Charlotte Home page: Office Hours Tuesday/Thursday: 9:30 am to 10:30 am 
Skype (will be on during office hours, username abwilkinson)
or in person in office
outline.26 26

27. UNC-Wilmington Clayton Ferner Department of Computer Science University of North Carolina Wilmington Home page: Office Hours Tuesday/Thursday TBD
outline.27 27

28. Office hours TBD, Woodward 208
Course Teaching Assistant
Yasaman Hessary
Office hours TBD, Woodward 208
Contact TA on matters such as user accounts, UNC-C system not functioning, or assignment grades if graded by TA, or general assistance. If a UNC-C system issue, also copy to
For issues with UNC-W system, contact Dr. Ferner.
Contact instructors for matters such as course materials or assignment write-up.
For issues with virtual machine, contact Dr. Wilkinson

29. Questions