1. Data Structures and Programming
資料結構與程式設計

2. Instructor: 顏嗣鈞 E-mail: hcyen@ntu.edu.tw
Web:
Time: 9:10-12:00 AM, Tuesdaay
Place: BL 113
Office hours: by appointment
Class web page:

3. 顏嗣鈞 學歷 博士 Univ. of Texas at Austin (計算機科學) 1986
碩士 交大計算機工程研究所
學士 台大電機系
經歷 台大電機系 教授 – present
台大計算機及資訊網路中心 主任 2014 – present
台大電機系 系主任
台大電機系 副教授
美國Iowa State Univ. 計算機科學系助理教授
專長 演算法設計分析、資訊視覺化、計算理論

4. PREREQUISITES
Familiarity in PASCAL, C, C++, or JAVA.

5. Textbook:
Data Structures & Algorithm Analysis in C++ (3nd or 4th Ed.),
Mark Weiss, Addison Wesley.

6. TOPICS lPRELIMINARIES: Introduction. Algorithm analysis.
lABSTRACT DATA TYPES: Stacks. Queues. Lists. List operations. List representations. List traversals. Doubly linked lists.
lTREES: Tree operations. Tree representations. Tree traversals. Threaded trees. Binary trees. AVL trees. 2-3 trees. B-trees. Red-black trees. Binomial trees. Splay trees, and more.
lHASHING: Chaining. Open addressing. Collision handling.
lPRIORITY QUEUES: Binary heaps. Binomial heaps. Fibonacci heaps. Min-max heaps. Leftist heaps. Skew heaps.
lSORTING: Insertion sort. Selection sort. Quicksort. Heapsort. Mergesort. Shellsort. Lower bound of sorting.
lDISJOINT SETS: Set operations. Set representations. Union-find. Path compression.
lGRAPHS: Graph operations. Graph representations. Basic graph algorithms.
lAMORTIZED ANALYSIS. Binomial heaps, Skew heaps. Fibonacci heaps.
lADVANCED DATA STRUCTURES: Tries. Top-down splay trees, and more.

7. Grading Homework + Prog. Assignments: 25-30% Midterm exam.: 35-40%
Final exam.: %

8. Academic Integrity
With the exception of group assignments, the work (including homework, programming assignments, tests) must be the result of your individual effort. This implies that one student should never have in his/her possession a copy of all or part of another student's homework. It is your responsibility to protect your work from unauthorized access. Academic dishonesty has no place in a university, in particular, in NTUEE. It wastes our time and yours, and it is unfair to the majority of students. Any form of cheating will automatically result in a failing grade in the course.

9. Data Structures vs. Programming
Algorithms
Programs
Data Structures + =

10. C++  Data Structures
One of the all time great books in computer science:
The Art of Computer Programming ( )
by Donald Knuth
Examples in assembly language (and English)!
American Scientist
says: in top 12 books
of the CENTURY!

11. Abstract Data Types Data Types integer, array, pointers, … Algorithms
Abstract Data Type (ADT)
Mathematical description of an object and the set of operations on the object
tradeoffs!
Given that this is computer science, I know you’d be disappointed if there were no acronyms in the class. Here’s our first one!
Now, what an ADT really is is the interface of a data structure without any specification of the implementation.
In this class, we’ll study groups of data structures to implement any given abstract data type. In that context…
Data Types
integer, array, pointers, …
Algorithms
binary search, quicksort, …

12. Advanced Data Structures
“Why not just use a big array?”
Example problem
Search for a number k in a set of N numbers
Solution # 1: Linear Search
Store numbers in an array of size N
Iterate through array until find k
Number of checks
Best case: 1 (k=15)
Worst case: N (k=27)
Average case: N/2 ?
Sorted array

13. Advanced Data Structures
Solution # 2: Binary Search Tree (BST)
Store numbers in a binary search tree
Requires: Elements to be sorted
Properties:
The left subtree of a node contains only nodes with keys less than the node's key
The right subtree of a node contains only nodes with keys greater than the node's key
Both the left and right subtrees must also be binary search trees
Search tree until find k
Number of checks
Best case: 1 (k=15)
Worst case: log2 N (k=27)
Average case: (log2 N) / 2 15 22 27 19 10 12 3

14. Should things be “Ordered”?

15. Example Does it matter? Problem Artifacts
N = 1,000,000,000
1 billion (Walmart transactions in 100 days)
1 Ghz processor = 109 cycles per second
Solution #1 ( assume 10 cycles per check)
Worst case: 1 billion checks = 10 seconds
Solution #2 (assume 10 cycles per check)
Worst case: 30 checks = seconds

16. Analysis
Does it matter?
N vs. (log2 N)

17. Computational Complexity
Computational complexity: an abstract measure of the time and space necessary to execute an algorithm as functions of its “input size”.
Input size: size of encoded “binary” strings.
sort n words of bounded length - input size: n
the input is the integer n - input size: lg n
the input is the graph G(V, E) - input size: |V| and |E|
Runtime comparison: assume 1 BIPS,1 instruction/op.
Time
Big-Oh
n = 10
n = 100
n = 104
n = 106
n = 108
500
O(1)
5*10-7 sec 3n
O(n)
3*10-8 sec
3*10-7 sec
3*10-5 sec
0.003 sec
0.3 sec
n lg n
O(n lg n)
6*10-7 sec
1*10-4 sec
0.018 sec
2.5 sec n2
O(n2)
1*10-7 sec
1*10-5 sec
0. 1 sec
16.7 min
116 days n3
O(n3)
1*10-6 sec
0.001 sec
31.7 yr ∞ 2n
O(2n)
4 *1011 cent. n!
O(n!)
Spring 2013

18. Can’t Finish the Assigned Task
“I can’t find an efficient algorithm, I guess I’m just too dumb.”

19. Mission Impossible
“I can’t find an efficient algorithm, because no such algorithm is possible.”

20. 愛斯坦
諾爾
“I can’t find an efficient algorithm, but neither can all these famous people.”

21. Motivating Example: Minimum Spanning Tree
Given an undirected graph G = (V, E) with weights on the edges, a minimum spanning tree (MST) of G is a subset T  E such that
T is connected and has no cycles,
T covers (spans) all vertices in V, and
sum of the weights of all edges in T is minimum.

22. Prim's MST Algorithm

23. Another Example: Shortest Path
Given a weighted graph and two vertices u and v, we want to find a path of minimum total weight between u and v.
Length of a path is the sum of the weights of its edges.
Applications: Internet packet routing, Flight reservations, Driving directions
ORD
PVD
MIA
DFW
SFO
LAX
LGA
HNL
849
802
1387
1743
1843
1099
1120
1233
337
2555
142
1205

24. Dijkstra’s algorithm C B A E D F 3 2 8 5 4 7 1 9 A 4 8 2 8 2 4 7 1 B CC B A E D F 3 2 8 5 4 7 1 9 A 4 8 2 8 2 4 7 1 B C D 3 9   2 5 E F A 4 A 4 8 8 2 2 8 2 3 7 2 3 7 1 7 1 B C D B C D 3 9 3 9 5 11 5 8 2 5 2 5 E F E F

25. Example (cont.) A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F A 4 8 2 7 2 3A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F

26. Questions Operations performed? Find neighbors Find/remove minimumA
Find neighbors 4 8 2 8 2 4 B C D 2 8 4 B C D
Find/remove minimum   E F 8 2 3 7 1 B C D
Update neighbors 3 9 5 11 2 5 E F

27. Key steps Find/remove minimum Update 2 C 2 C 8 8 B B  4  4  D F  D3 8 D B 5 E 11 F

28. Straightforward approachB C D E F 8 2 4 
Find min: scan thru the array; Remove item B D E F 8 3 5 11
Update: a left-right scan again
Questions:
Is the above efficient?
Can we do better?

29. Priority Queues Heaps Operation Linked List Binary Binomial
Fibonacci *
Relaxed
make-heap 1 1 1 1 1
insert 1
log N
log N 1 1
find-min N 1
log N 1 1
delete-min N
log N
log N
log N
log N
union 1 N
log N 1 1
decrease-key 1
log N
log N 1 1
delete N
log N
log N
log N
log N
not possible to get all ops O(1) if only use comparisons because of N log N sorting lower bound
Brodal gives variant of relaxed heap that achieves worst-case bounds for all operations
relaxed heap: relaxes heap ordering property
is-empty 1 1 1 1 1
O(|V|2)
O(|E| log |V|)
O(|E| + |V| log |V|)
Dijkstra/Prim
1 make-heap
|V| insert
|V| delete-min
|E| decrease-key

30. Life Cycle of Software Development

31. Is this program correct? —How do we know?
int Find(float[] a, int m, int n, float x) {
while (m < n) {
int j = (m+n) / 2;
if (a[j] < x) { m = j+1; } else if (x < a[j]) { n = j-1; } else { return j; } } return -1; }

32. Making sense of programs
Program semantics defines programming language
e.g., Hoare logic, Dijkstra's weakest preconditions
Specifications record design decisions
bridge intent and code
Tools amplify human effort
manage details
find inconsistencies
ensure quality

33. Program Correctness (Example)
(x != null ==> x != null && x.f >= 0) && (x == null ==> z-1 >= 0)
if (x != null) { n = x.f; } else { n = z-1; z++; } a = new char[n];
x != null && x.f >= 0
z-1 >= 0
n >= 0
true

34. Data “Structures” General areas include: ● Sequential storage
● Hierarchical storage
● Adjacency storage

35. Goal Learn to write efficient and elegant software
How to choose between two algorithms
Which to use? bubble-sort, insertion-sort, merge-sort
How to choose appropriate data structures
Which to use? array, vector, linked list, binary tree

36. Why should you care?
Complex data structures and algorithms are used in every real program
Data compression uses trees: MP3, Gif, etc…
Networking uses graphs: Routers and telephone networks
Security uses complex math algorithms: GCD and large decimals
Operating systems use queues and stacks: Scheduling and recursion
Many problems can only be solved using complex data structures and algorithms

37. In this course, we will look at:
different techniques for storing, accessing, and modifying information on a computer
algorithms which can efficiently solve problems
We will see that all data structures have trade-offs – there is no ultimate data structure...
The choice depends on our requirements

38. What this course is NOT about
This course is not about C++
Although we will use C++ to implement some of the concepts
This course is not about MATH
Although we will use math to formalize many of the concepts
Competency in both math and C++ is therefore welcomed.
C++: inheritance, overloading, overriding, files, linked-lists, multi-dimensional arrays
Math: polynomials, logarithms, inductive proofs, logic

39. The Big Idea Definition of Abstract Data Type
A collection of data along with specific operations that manipulate that data
Has nothing to do with a programming language!
Two fundamental goals of algorithm analysis
Correctness: Prove that a program works as expected
Efficiency: Characterize the run-time of an algorithm

40. The Big Idea Alternative goals of algorithm analysis
Characterize the amount of memory required
Characterize the size of a programs code
Characterize the readability of a program
Characterize the robustness of a program

41. Clever? Efficient? Data Structures Algorithms Insert
Delete
Find
Merge
Shortest Paths
Union
Lists, Stacks, Queues
Heaps
Binary Search Trees
AVL Trees
Hash Tables
Graphs
Disjoint Sets
Data Structures
Algorithms

42. Why study data structures?
Clever ways to organize information in order to enable efficient computation
Databases AI
Theory
Graphics
Networking
Games
Clever – range from techniques with which you are already familiar – eg, representing simple lists – to ones that are more complex, such as hash tables or self-balancing trees. Elegant, mathematically deep, non-obvious.
Making the different meanings of “efficient” precise is much of the work of this course!
Systems
Applications
Data
Structures