1. CSE 2001: Introduction to the Theory of Computation
Welcome
Jeff Edmonds
CSB 3044, ext  
Jeff Edmonds
York University
Lecture 1
COSC 2001

2. I request that you do everything in your power.
A Contract to Learn
It deeply saddens me when a third of the class does not learn the material sufficiently to pass.
I will do everything in my power to help you learn this material.
I request that you do everything in your power.
Everyone can learn it.

3. Tough Love Now, I threatened them every day.
They performed better than ever before!
And my course evaluations were better.

4. Study Now! You spend 50hrs/week on your programming course.
People do work hard on my assignments.
But they don’t really study the material deeply until they fail the midterm.

5. Together
I think it is important for people to not feel isolated with the material.

6. Together Yes! The best way to learn is to teach each other!
You’re cool! Are you free sometime this weekend?
Yes! The best way to learn is to teach each other!

7. Together Some students feel too intimidated to talk to the professor.
Actually, he is just a guy who has been doing this for a while.

8. Together
Office hours: Before/after class tend to be the best time. Longer or private questions, when requested, can be taken back to my office, CSB 3044. 8

9. Material

10. Material

11. Notes I wrote before I made the slides. (Slides were made from them)
Material
Notes I wrote before I made the slides. (Slides were made from them)

12. Material

13. Marks

14. Marks

15. Marks Class Participation: Asking/Answering questions in class
Attending office hours
Talking to me outside of class
Submitting a photo *

16. Useful Learning Techniques16

17. Read Ahead
You are expected to read the lecture notes before the lecture.
This will facilitate more productive discussion during class.
Also please proof read assignments & tests.
Like in an English class

18. Explaining
We are going to test you on your ability to explain the material.
Hence, the best way of studying is to explain the material over and over again out loud to yourself, to each other, and to your stuffed bear.

19. Day Dream While going along with your day
Mathematics is not all linear thinking.
Allow the essence of the material to seep into your subconscious
Pursue ideas that percolate up and flashes of inspiration that appear.

20. Ask questions. Why is it done this way and not that way?
Be Creative
Ask questions.
Why is it done this way and not that way?

21. Useful Learning Techniques
Before reading the solutions to the assignments.
Work hard on solving them on your on.
Study the text and slides.
Get help, but not answers from friends, TA, professor, …

22. Useful Learning Techniques
After reading the solutions to the assignments.
Study the solutions.
Understand the solutions.
Memorize the solutions.
The questions on the tests will be different.
But the answers will be surprisingly close.

23. Useful Learning Techniques
Please interact with me in class.
Help me know what people are not understanding
Slow down the slides
(Though we do have a lot of material to cover)

24. Useful Learning Techniques
Please ask questions!

25. Useful Learning Techniques
Please ask questions!
To keep the flow going
Wiggly hand: relevant to current slide.
Stationary hand: question about past material.

26. Useful Learning Techniques
When I ask a question to the class.
Please don’t shout out answers. So others can think.

27. Useful Learning Techniques
In every class, there is one student I don’t like.
The key when talking in class is
Are you trying to help me and the class or win some compitition?

28. Useful Learning Techniques
Please ask questions!
Winter 06, the average of one section was much higher than the average of the other.   (I taught them both)
My theory was that it was because a student, Gertruda, constantly asked great questions and everyone learned from them.
Ask questions for everyone’s sake.
10% of your mark is class participation!

29. May I have a letter of reference for grad school?
Two years from now:
May I have a letter of reference for grad school?
I find it awkward to write letters for people that I don’t recognize.
Make yourself KNOWN today to SOME professor.

30. May I have a letter of reference for grad school?
Two years from now:
May I have a letter of reference for grad school?
I find it awkward to write letters for people that I don’t recognize.
Make yourself KNOWN today to SOME professor.
Sorry does not help as I don’t remember names

31. Submit a Photo What: Submit a photo on line
Add a photo to your forum account
If you would like a better rapport with your s, please include a photo.
Why:
I am very bad with names. (Sorry)
s and tests are effectively anonymous.
Needed for 10% participation mark.
You can see the class list.
How:
Submit photo (named as your name appears in class list)
     - cp photo.jpg Smith_John.jpg 
- submit 3101 photos Smith_John.jpg
Forum
- Upload a photo to your Forum Avatar. 31

32. Forum I communicate to you via the Forum.
You are also encouraged to talk to each other this way.
To be sure that you know when something has been posted, it is important to subscribe to the course forum.
Be sure to subscribe to the course forum and not just to one of its topics 32

33. Introduction to Theory of Computation
Thinking Abstractly
Introduction to Theory of Computation
I have complained for years. This should not be a second year course.
It teaches theory is hard and useless!!!
But it is useful for teaching

34. Devastated by the midterm
Thinking Abstractly
This course requires completely changing the way you think about computation.
Though I keep warning people,
they tend not to get it until they are
Devastated by the midterm
Change your thinking now.

35. Thinking Abstractly
For the people who can’t, the material is impossible.
For the people who can think abstractly the material is easy.

36. Thinking Abstractly
The psychological profiling of a successful person is mostly the ability to shift levels of abstraction, from low level to high level.
To understand the detailed workings.
To understand the big picture.
Donald Kunth

37. Thinking Abstractly
The psychological profiling of a successful person is mostly the ability to shift levels of abstraction, from low level to high level.
To understand the detailed workings.
To understand the big picture.
To understand complex things in simple ways.
Donald Kunth =

38. It is hard to think of love in terms of the firing of neurons.
Thinking Abstractly vs
It is hard to think of love in terms of the firing of neurons.
Software developers view subsystems as entities with separate personalities, roles, and interactions,
not details of code. vs

39. Thinking Abstractly
Better for communicating to friend

40. Thinking Abstractly
A form that can peculate down and dwell in your subconscious
where the miracle leaps of inspiration happen
Your subconscious does not understand JAVA code.

41. Thinking Abstractly Paradigm Shift
Is the black the form and the green the background?
Is the green the form and the black the background?
It is helpful to have different ways of looking at it.

42. Different Representations of Algorithms
Code
Running example
Turing Machines and Finite Automata
Context Free Grammars

43. Code Representation of an Algorithm
class InsertionSortAlgorithm extends SortAlgorithm {
void sort(int a[]) throws Exception {
for (int i = 1; i < a.length; i++) {
int j = i;
int B = a[i];
while ((j > 0) && (a[j-1] > B)) {
a[j] = a[j-1];
j--; }
a[j] = B; }}
Pros and Cons?

44. Code Representation of an Algorithm
Pros:
Cons:
Runs on computers
Precise and succinct
Perception that being able to code is the only thing needed to get a job. (false)
I am not a computer
I need a higher level of intuition.
Prone to bugs
Language dependent

45. Running Example Representation of an Algorithm
Try out a problem or solution on small examples.

46. Running Example Representation of an Algorithm88 88 52 98 14 14 31 98 25 30 62 23 79

47. Running Example Representation of an Algorithm
14,23,25,30,31,52,62,79,88,98
Pros and Cons?

48. Running Example Representation of an Algorithm
Pros:
Cons:
Concrete
Dynamic
Visual
Relies on you to find the pattern.
Does not explain why it works.
Demonstrates for only one of many inputs.

49. Turing Machines Representation of an Algorithm
A Turing machine
Is historically the first model of computation.
It is abstract.
It is NOT practical.
We will try to give it “meaning”.
Instead of trying to design TMs, we will compile any Java like program into a TM.

50. Finite Automata Representation of an Algorithm
The input arrives as a stream. One character at a time. It cannot be reread.
Eg:
simple iterative algorithms
simple mechanical or electronic devices like elevators and calculators
simple processes like the job queue of an operating system
simple patterns within strings of characters.
simple languages of strings
{01,100,01110,… }

51. Finite Automata Representation of an Algorithm
The input arrives as a stream. One character at a time. It cannot be reread.
Pros and Cons?

52. DFA and Turing Machines Representation of an Algorithm
Pros:
Cons:
Good for theorems about algorithms
Having a different way of thinking of algorithms, provides a deeper understanding.
Mathematical mumbo jumbo
Too abstract
Students resist it

53. Representation of an Algorithm
Regular Expression
Representation of an Algorithm
Regular Expression are a quick notation for defining a language of strings.
Any finite set of finite strings is a regular expression
Eg R = {0,01,11}, then L(R) = {0,01,11}.
R1R2 , R1∙R2 , and R*
representing L1L2 , L1∙L2 , and L*
Extended Regular Expressions
Also R1R2 and R

54. Complexity Classes Linear DFA=NFA =Extended Regular =Regular
These languages are called Regular Languages.
Linear
Problems which have
TM/Java programs that solve them in linear time
DFA=NFA
=Extended Regular
=Regular

55. Context Free Grammar Representation of an Algorithm

56. Context Free Grammar Representation of an Algorithm
Pros:
Cons:
Adds “meaning” to the string.
Can be used to evaluate/process the string.
Mircosoft Word underlines poor grammar with green.
Used in a compiler to produce machine code.
Mathematical mumbo jumbo
Too abstract
Students resist it

57. Hierarchy of Infinities=
<<
|Integers|
|Fractions|
|Reals|
Each defined by an infinite string
Each defined by a finite string
Each defined by a finite string

58. Hierarchy of Infinities=
|Integers|
|Fractions|
...
... 6
6/1
6/2
6/3
6/4
6/5
6/6
6/7
6/8
All positive fractions 5
5/1
5/2
5/3
5/4
5/5
5/6
5/7
5/8 4
4/1
4/2
4/3
4/4
4/5
4/6
4/7
4/8 3
3/1
3/2
3/3
3/4
3/5
3/6
3/7
3/8
Count them
by mapping. 2
2/1
2/2
2/3
2/4
2/5
2/6
2/7
2/8
... 1
1/1
1/2
1/3
1/4
1/5
1/6
1/7
1/8
Every rational gets mapped to some integer!
{ 1, 2, 3, 4, 5, 6, …. }
All positive integers

59. Hierarchy of Infinities
<<
|Fractions|
|Reals|
Proof that |R| > |N|
Given a list of all real numbers … … … … …
…..
Proof by Diagonalization
We find a real number xdiagonal that is not in the list.
xdiagonal = …

60. History of Computability
Hilbert (1900)
Can every problem be computed?
Or are some Uncomputable?
Need to formally define
“Algorithm”

61. History of Computability
Computable
Turing (1936)
A problem is Computable
if it can be computed by a Turing Machine. (1936)
Need to formally define
“Algorithm”

62. History of Computability
Computable
Turing (1936)
Equivalent via compiler to:
Java:
Need to formally define
“Algorithm”

63. History of Computability
Halting
Problem
Turing proves that the halting problem is NOT Computable!
Computable
Turing (1936)

64. History of Classifying Problems
Halting
Computable
Exp
Poly
Jack Edmonds 1965
Known
GCD

65. History of Classifying Problems
Halting
Computable
Jack Edmonds
Steve Cook NP
exponential time to search
poly time to verify given witness
Non-Deterministic Polynomial Time
Exp
Poly
SAT
Known
Matching
Circuit-Sat Problem:
Does a circuit have a satisfying assignment.
GCD

66. History of Classifying Problems
Halting
Computable
Jack Edmonds
Steve Cook NP
exponential time to search
poly time to verify given witness
Exp
Poly
Non-Deterministic Polynomial Time
SAT
Known
Matching
Circuit-Sat Problem:
Does a circuit have a satisfying assignment.
GCD

67. History of Classifying Problems
Steve Cook 1971
Halting
Computable
Exp
Poly
Games NP
SAT
complete
poly Prob’
SAT
Let’s compare the problems in NP
None are harder than SAT!
Known
Matching
Circuit-Sat Problem:
Does a circuit have a satisfying assignment.
GCD

68. History of Classifying Problems
Given an oracle for SAT, how can you quickly solve other problems?
SAT Oracle
Other Problem

69. History of Classifying Problems
Design an alg.
Assume have an alg.
SAT Oracle
Other Problem ≤
Proof by Reductions

70. History of Classifying Problems
Other Input
Other Output
SAT Oracle
Other Problem
SAT Input
SAT Output ≤
Proof by Reductions

71. Gödel’s Incompleteness Theorem
Some things have no proof and whether true or false depends on your world view.
For every proof system S, there are math statements Φ which are either not proved or proved incorrectly!

72. Abstract away the inessential features of a problem
Value Simplicity
Abstract away the inessential features of a problem =
Goal: Understand and think about complex algorithm in simple ways.
Don’t tune out. There are deep ideas within the simplicity.

73. End