1. MapReduce, Dictionaries, List Comprehensions
Special thanks to Scott Shawcroft, Ryan Tucker, and Paul Beck for their work on these slides.
Except where otherwise noted, this work is licensed under:

2. Why didn't NaN == NaN? Last time we found that
float(“nan”) == float(“nan”) is False
NaN is “nothing”, does not equal anything, even itself.
To find out if a value is NaN you have to use isnan
>>> from math import *
>>> isnan(n)
True
>>> n = float("nan")

3. MapReduce
Framework for processing huge datasets on certain kinds of distributable problems
Map Step:
- master node takes the input, chops it up into smaller sub-problems, and distributes those to worker nodes.
- worker node may chop its work into yet small pieces and redistribute again

4. MapReduce
Framework for processing huge datasets on certain kinds of distributable problems
Map Step:
- master node takes the input, chops it up into smaller sub-problems, and distributes those to worker nodes.
- worker node may chop its work into yet small pieces and redistribute again

5. MapReduce Reduce Step:
- master node then takes the answers to all the sub-problems and combines them in a way to get the output

6. MapReduce Problem: Given an email how do you tell if it is spam?
- Count occurrences of certain words. If they occur too frequently the is spam.

7. MapReduce 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
= ['the', 'this', 'annoy', 'the', 'the', 'annoy']
>>> def in (x):
if (x == "the"):
return 1;
else:
return 0;
>>> map (in , l)
[1, 0, 0, 0, 1, 1, 0]
>>> reduce ((lambda x, xs: x + xs), map(in , ))

8. Stack Information found here: Really easy!
html
Really easy!
Use lists to simulate a stack
.append(value) in order to “push” onto the stack
.pop() in order to ‘pop’ off the stack.
Check this example out!
>>> a = []
>>> a.append(1)
>>> a.append(2)
>>> a.append(3)
>>> a.append(5)
>>> a
[1, 2, 3, 5]
>>> a.pop() 5
[1, 2, 3] 3
[1, 2] 8

9. Queue Really easy too!!!! Use lists to simulate a queue
.insert(0, value) in order to ‘add’ to the queue.
.pop() in order to ‘remove’ off from the queue.
Why does this work? What do insert and pop really do?
>>> a = []
>>> a.insert(0,2)
>>> a.insert(0,3)
>>> a.insert(0,4)
>>> a
[4, 3, 2]
>>> a.pop() 2
[4, 3]
>>> a.insert(0, a.pop()) 9

10. Queue Ok… this is dumb… and inefficient.
Using lists, we get O(N), but…..
from collections import deque
queue = deque([‘blah’, ‘blah’, ‘blah’])
Use append() to ‘add’ to the queue
Use popleft() to ‘remove’ from the queue
This works in O(1)!
Sweeeeeeeeet.
>>> from collections import deque
>>> my_list = ["Jordan", "Zeratul", "Kerrigan", "Raynor", "Artosis"]
>>> my_list
['Jordan', 'Zeratul', 'Kerrigan', 'Raynor', 'Artosis']
>>> q = deque(my_list)
>>> q
deque(['Jordan', 'Zeratul', 'Kerrigan', 'Raynor', 'Artosis'])
>>> q.append("Tasteless")
deque(['Jordan', 'Zeratul', 'Kerrigan', 'Raynor', 'Artosis', 'Tasteless'])
>>> q.append("IMMVP" )
deque(['Jordan', 'Zeratul', 'Kerrigan', 'Raynor', 'Artosis', 'Tasteless', 'IMMVP'])
>>> q.popleft()
'Jordan'
'Zeratul' 10

11. Dictionary Equivalent to Java’s Map. Stores keys and values.
Empty dictionary is defined by:
m = {}
Pre-populated dictionary: (Map<String, String>)
(mapping names to SCII division…)
m2 = { ‘jordan’ : ‘master’, ‘roy’ : ‘bronze’, ‘marty’ : ‘bronze’ }
Add to dictionary:
m2[ ‘IMMVP’ ] = ‘master’
Retrieve from dictionary
m2[‘jordan’] # ‘master’ 11

12. Dictionary How to get keySet() map.keys()
Uh oh……. We get something called dict_keys…
We should cast to a list.
list(map.keys())
How to check if a key is contained in a dictionary…
‘marty’ in m # returns true bc ‘marty’ is in dictionary
‘bronze’ in m # returns false bc ‘bronze’ is not a key
How to delete
del m[‘marty’]
Get values with: m.values()
Note that you get the dict_values. So cast to a list.
>>> m
{'hi': 3, 'hello': 32}
>>> m = {'jordan' : 'master', 'roy' : 'bronze', 'marty' : 'bronze' }
{'marty': 'bronze', 'roy': 'bronze', 'jordan': 'master'}
>>> m.keys()
dict_keys(['marty', 'roy', 'jordan'])
>>> a = list(m.keys())
>>> a
['marty', 'roy', 'jordan']
>>> 'marty' in m
True
>>> 'roy' in m
>>> 'oy' in m
False 12

13. Set s = { ‘GLaDOS’, ‘Cloud’, ‘Aeris’, ‘Shepard’, ‘Cloud’, ‘Aeris’ }
Removed duplicates
s2 = set(‘hello I am soooo happy!!!’)
Creates a set out of all the characters
The stuff in the blue must be iterable.
i.e. lists, tuples, etc…
Check for contains is same as dictionary.
‘GLaDOS’ in s
>>> s = {'a', 'b', 'c', 'a', 'b', 'd'}
>>> s
{'a', 'c', 'b', 'd'}
>>> 'b' in s
True
>>> a = set('hello world i am soooooo happy')
>>> a
{'a', ' ', 'e', 'd', 'i', 'h', 'm', 'l', 'o', 'p', 's', 'r', 'w', 'y'}
>>> 13

14. Set Tons of extra functionality
Check for subsets, see if set A is contained in set B
Go here to see more: (under 5.7) 14

15. List Comprehensions Applies the expression to each element in the list
[expression for element in list]
Applies the expression to each element in the list
You can have 0 or more for or if statements
If the expression evaluates to a tuple it must be in parenthesis 15

16. List Comprehensions 1 2 3 4 5 6 7 8 9 10 11 12 13 14
>>> vec = [2, 4, 6]
>>> [3*x for x in vec]
[6, 12, 18]
>>> [3*x for x in vec if x > 3]
[12, 18]
>>> [3*x for x in vec if x < 2] []
>>> [[x,x**2] for x in range(10)]
[[0, 0], [1, 1], [2, 4], [3, 9]]
>>> [x, x**2 for x in vec]
# error - parens required for tuples 16

17. List Comprehensions
You can do most things that you can do with map, filter and reduce more nicely with list comprehensions
How can we find out how many times ‘a’ appears in the list named ? 1 2 3 4 5 6
>>> = ['once', 'upon', 'a', 'time', 'in', 'a', 'far', 'away']
>>> len( [1 for x in if x == 'a'] )
>>> 2
Let’s explain this plz…. 17