1. Insertion sort Loop invariants Dynamic memory
Correctness
Insertion sort
Loop invariants
Dynamic memory
September 19, 2017
Cinda Heeren / Geoffrey Tien

2. Cinda Heeren / Geoffrey Tien
Insertion sort
Another simple sorting algorithm
Divides array into sorted and unsorted parts
The sorted part of the array is expanded one element at a time
Find the correct place in the sorted part to place the 1st element of the unsorted part
By searching through all of the sorted elements
Move the elements after the insertion point up one position to make space
September 19, 2017
Cinda Heeren / Geoffrey Tien

3. Cinda Heeren / Geoffrey Tien
Insertion sort 23 41 33 81 07 19 11 45
treats first element as sorted part 23 41 33 81 07 19 11 45
locate position for comparison 23 33 41 81 07 19 11 45
locate position for comparisons 23 33 41 81 07 19 11 45
locate position for comparison 07 23 33 41 81 19 11 45
locate position for comparisons 07 19 23 33 41 81 11 45
locate position for comparisons 07 11 19 23 33 41 81 45
locate position for comparisons 07 11 19 23 33 41 45 81
locate position for comparisons
September 19, 2017
Cinda Heeren / Geoffrey Tien

4. Insertion sort algorithm
void InsertionSort(int arr[], int size) {
for (int i = 1; i < size; ++i)
temp = arr[i];
int pos = i;
// Shuffle up all sorted items > arr[i]
while (pos > 0 && arr[pos - 1] > temp)
arr[pos] = arr[pos – 1];
pos--; }
// Insert the current item
arr[pos] = temp;
Outer loop: 𝑛 times
Inner loop: how many times?
Minimum: one comparison per outer loop iteration, 𝑛 times
Maximum: 𝑖−1 comparisons per outer loop iteration, 𝑛 𝑛−1 2 times
September 19, 2017
Cinda Heeren / Geoffrey Tien

5. Cinda Heeren / Geoffrey Tien
Insertion sort cost
Sorted
Elements
Worst-case Search
Worst-case Shuffle 1 2 …
𝑛−1
𝑛(𝑛−1)/2
September 19, 2017
Cinda Heeren / Geoffrey Tien

6. Insertion sort best case
The efficiency of insertion sort is affected by the state of the array to be sorted
In the best case the array is already completely sorted!
No movement of array elements is required
Requires 𝑛 comparisons
September 19, 2017
Cinda Heeren / Geoffrey Tien

7. Insertion sort worst case
In the worst case the array is in reverse order
Every item has to be moved all the way to the front of the array
The outer loop runs 𝑛−1 times
In the first iteration, one comparison and move
In the last iteration, 𝑛−1 comparisons and moves
On average, 𝑛/2 comparisons and moves
For a total of 𝑛 ∗ (𝑛−1) / 2 comparisons and moves
September 19, 2017
Cinda Heeren / Geoffrey Tien

8. Insertion sort average case
What is the average case cost?
Is it closer to the best case?
Or the worst case?
If random data is sorted, insertion sort is usually closer to the worst case
Around 𝑛 ∗ (𝑛−1) / 4 comparisons
September 19, 2017
Cinda Heeren / Geoffrey Tien

9. Analyzing loops Loop invariants
Proving algorithm correctness using loop invariants
Loop invariants
Properties of the algorithm or structure that are always true (i.e. do not vary) at particular points in the program
Within a loop body, these properties may become violated briefly, but the rest of the loop instructions should fix the properties for the next iteration
E.g. we have an algorithm for Insertion sort. How can we prove that it really works for all possible input configurations?
September 19, 2017
Cinda Heeren / Geoffrey Tien

10. Loop invariants Outer loop invariant
Example: Correctness of insertion sort
void InsertionSort(int arr[], int size) {
for (int i = 1; i < size; i++) {
int temp = arr[i];
int pos = i;
while (pos > 0 && arr[pos-1] > temp) { // shuffle up sorted
arr[pos] = arr[pos-1]; // items > arr[i]
pos--; }
arr[pos] = temp; // insert current item into empty space
Outer loop invariant
Before each test 𝑖<size (including the last one), the elements in array[0,…𝑖−1] are in sorted order
September 19, 2017
Cinda Heeren / Geoffrey Tien

11. Proving a loop invariant
Example: Correctness of insertion sort
Induction variable
Number of times through the loop
Base case (initialization)
Prove the invariant true before the first loop guard test
Induction hypothesis
Assume the invariant holds just before some (unspecified) iteration's loop guard test
Inductive step (maintenance)
Prove the invariant holds at the end of that iteration (just before the next loop guard test
Termination
Ensure that the loop terminates, and the invariant property at termination helps establish the correctness of the algorithm
September 19, 2017
Cinda Heeren / Geoffrey Tien

12. Insertion sort correctness
void InsertionSort(int arr[], int size) {
for (int i = 1; i < size; i++) {
int temp = arr[i];
int pos = i;
while (pos > 0 && arr[pos-1] > temp) { // shuffle up sorted
arr[pos] = arr[pos-1]; // items > arr[i]
pos--; }
arr[pos] = temp; // insert current item into empty space
Outer loop invariant:
Before i < size, arr[0…i-1] is sorted
Base case, i == 1
arr[0…0] has only one element, so it is always in sorted order
Note: this is only for arrays with at least 1 element
September 19, 2017
Cinda Heeren / Geoffrey Tien

13. Insertion sort correctness
void InsertionSort(int arr[], int size) {
for (int i = 1; i < size; i++) {
int temp = arr[i];
int pos = i;
while (pos > 0 && arr[pos-1] > temp) { // shuffle up sorted
arr[pos] = arr[pos-1]; // items > arr[i]
pos--; }
arr[pos] = temp; // insert current item into empty space
Outer loop invariant:
Before i < size, arr[0…i-1] is sorted
Induction hypothesis
Just before we test k < size, arr[0…k-1] are in sorted order
When the loop starts, i == k
Inductive step
Inner loop shuffles up elements larger than arr[i] and puts arr[i] into place, so arr[0…i] is in sorted order
Assuming the inner loop works correctly!
September 19, 2017
Cinda Heeren / Geoffrey Tien

14. Insertion sort correctness
void InsertionSort(int arr[], int size) {
for (int i = 1; i < size; i++) {
int temp = arr[i];
int pos = i;
while (pos > 0 && arr[pos-1] > temp) { // shuffle up sorted
arr[pos] = arr[pos-1]; // items > arr[i]
pos--; }
arr[pos] = temp; // insert current item into empty space
Outer loop invariant:
Before i < size, arr[0…i-1] is sorted
Termination
Loop ends when i == size
This will happen since size is non-negative and i increases
By the loop invariant, arr[0…i-1] is sorted, and since i == size, arr[0…i-1] is arr[0…size-1] which is the whole array
September 19, 2017
Cinda Heeren / Geoffrey Tien

15. Exercise Loop invariants for Selection sort
void SelectionSort(int arr[], int size) {
for (int i = 0; i < size -1; ++i)
int smallest = i;
// Find the index of the smallest element
for (int j = i + 1; j < size; ++j)
if (arr[j] < arr[smallest])
smallest = j; }
// Swap the smallest with the current item
temp = arr[i];{
arr[i] = arr[smallest];
arr[smallest] = temp;
Outer loop puts the smallest item of the unsorted portion into the right side of the sorted portion
Inner loop finds the index of the smallest item
September 19, 2017
Cinda Heeren / Geoffrey Tien

16. Variables and memory in C++
Variables and arrays are allocated space in call stack
At least as they have been discussed so far, and
Assuming that they were declared in a function
Variables allocated space on the call stack are not permitted to change size
Stack memory is allocated in sequence and other variables might be over-written
September 19, 2017
Cinda Heeren / Geoffrey Tien

17. Cinda Heeren / Geoffrey Tien
Dynamic memory
aka heap memory
What happens if we want to allocate memory at run time?
It can’t be allocated to stack memory so will need to be put somewhere else
Let’s call somewhere else the heap or the free store
We still need local variables that refer or point to the dynamically allocated memory
In C++ such variables are pointers
September 19, 2017
Cinda Heeren / Geoffrey Tien

18. Variables in dynamic memory
Create a variable to store an address
A pointer to the type of data to be stored
Addresses have a fixed size
If there is initially no address it should be assigned a special value (null)
Create new data in dynamic memory
This may be done when needed (i.e. at run time)
Assign the address of the data to the pointer
This involves more a more complex management system than using the call stack
September 19, 2017
Cinda Heeren / Geoffrey Tien

19. Creating an array in dynamic memory
Arrays created in dynamic memory are indexed just like other arrays
int* p_arr = new int[100]; for (int i=0; i<100; ++i) { p_arr[i] = i+1; }
Pointers to arrays can be assigned new arrays
delete[] p_arr; //release memory
p_arr = new int[ ];
September 19, 2017
Cinda Heeren / Geoffrey Tien

20. Cinda Heeren / Geoffrey Tien
A dynamic array
int* seq = NULL;
double some_val = 2.397;
seq = Sequence(1, 3);
// Returns pointer to array {start, // start+1, … start+n-1}
int* Sequence(int start, int n) {
int* result = new int[n];
for(int i=0; i < n; i++) {
result[i] = start + i; }
return result;
main memory
2a34 is the main memory address of the array
stack
heap
2a34
2.397 1 2 3
seq
some_val
Builds array in dynamic storage (heap, free store)
September 19, 2017
Cinda Heeren / Geoffrey Tien

21. Cinda Heeren / Geoffrey Tien
A dynamic array
int* seq = NULL;
double some_val = 2.397;
seq = Sequence(1, 3);
seq = Sequence(4, 5);
// Returns pointer to array {start, // start+1, … start+n-1}
int* Sequence(int start, int n) {
int* result = new int[n];
for(int i=0; i < n; i++) {
result[i] = start + i; }
return result;
stack
heap
47b1
2a34
2.397 1 2 3
memory leak!
seq
some_val 4 5 6 7 8
September 19, 2017
Cinda Heeren / Geoffrey Tien

22. Releasing Dynamic Memory
When a function call is complete its stack memory is released and can be re-used
Dynamic memory should also be released
Failing to do so results in a memory leak
It is sometimes not easy to determine when dynamic memory should be released
Data might be referred to by more than one pointer
Memory should only be released when it is no longer referenced by any pointer
Rule of thumb: if you new something, you should delete it (eventually)
September 19, 2017
Cinda Heeren / Geoffrey Tien

23. Stack memory VS heap memory
fast access
allocation/deallocation and space automatically managed
memory will not become fragmented
local variables only
limit on stack size (OS-dependent)
variables cannot be resized
Heap
variables accessible outside declaration scope
no (practical) limit on memory size
variables can be resized
(relatively) slower access
no guaranteed efficient use of space
memory management is programmer's responsibility
September 19, 2017
Cinda Heeren / Geoffrey Tien

24. Readings for this lesson
Koffman
Chapters 10.4, P.5
Epp
Chapter 5.5
Next class – Pointers (finally!)
Koffman – P.5, 4.5
September 19, 2017
Cinda Heeren / Geoffrey Tien