Dsa Term Paper

Theory of Algorithms and Data Structures Lecture 1: Introduction
Dr William Smith [e-mail]

Theory of Algorithms and Data Structures

Administrivia – A Brief History of TAD
One becomes two… Theory of Algorithms and Data Structures (TAD) 10 credits Algorithms and Data Structures (ADS) 20 credits 18 theory lectures 9*2 hour practicals (ADA) 90 min closed exam 3 week open exam (Sum 2-5) Practice 18 theory lectures 4*1 hour practicals (theory) 90 min closed exam Practical Programming Skills (PPS) 10 credits 10 theory lectures 9*2 hour practicals (Java) 9 hours labs Open exam 1 (Spr 2-6) Open exam 2 (Spr 6-Sum 1)
Lecture 1

Theory

Theory of Algorithms and Data Structures

Administrivia – Course Organisation

 18 Lectures (Week 2-10/Spring)  12:15 on Mondays, 12:15 on Fridays  4 Practicals (problem classes)  90 minute closed exam (Week 8/Summer)

 Course notes on website (slightly different to those used in lectures – gaps left for you to fill in answers to questions)  Complete versions will appear online after the lecture

Lecture 1

Theory of Algorithms and Data Structures

Administrivia – Practicals
 4 Practicals – problem classes  Every other week starting in week 3 or 4  4:15 Monday or 9:15 Tuesday  40 minutes working in groups, 10 minute mini exam  You will get back marked mini-exams after ~1 week Group 1 4:15 Monday Weeks 3,5,7,9 Group 3 4:15 Monday Weeks 4,6,8,10 Group 2 9:15 Tuesday Weeks 3,5,7,9 Group 4 9:15 Tuesday Weeks 4,6,8,10

See website to find out which group you are in
Lecture 1

Theory of Algorithms and Data Structures

How should you spend your time?
10 credit module = 100 hours  18 hours in lectures  4 hours in practicals  1.5 hours in closed exam  Leaves 76.5 hours for private study – think about how best to use this time

Lecture 1

Theory of Algorithms and Data Structures

How should you spend your time?
10 credit module = 100 hours  18 hours in lectures  4 hours in practicals  1.5 hours in closed exam  Leaves 76.5 hours for private study – think about how best to use this time Something like: A couple of hours before/after each lecture = 36 hours

+

A couple of hours before/after each practical = 8 hours

+

32.5 hours for revision/past papers

Lecture 1

Theory of Algorithms and Data Structures

How should you spend your time?
10 credit module = 100 hours  18 hours in lectures  4 hours in practicals  1.5 hours in closed exam  Leaves 76.5 hours for private study – think about how best to use this time Something like: A couple of hours before/after each lecture = 36 hours

+

A couple of hours before/after each practical = 8 hours

+

32.5 hours for revision/past papers

 Suggested reading at end of each lecture and on webpage (+ other resources)
Lecture 1

Theory of Algorithms and Data Structures

Course Plan
Three elements: 1. Tools necessary to analyse algorithms 2. A look at some common algorithms and data structures 3. A look at some common algorithm design strategies

Lecture 1

Theory of Algorithms and Data Structures

Course Plan
Three elements: 1. Tools necessary to analyse algorithms 2. A look at some common algorithms and data structures 3. A look at some common algorithm design strategies Course philosophy:  No code - language neutral – English/pseudocode/examples/intuition  Emphasise important applications of standard algorithms  Make you stop and think – arrive at ideas yourself (problem classes and questions in lectures)  Help teach each other – work in teams on practical problems – explaining an idea to someone helps you solidify your own understanding  You will be expected to apply your knowledge (exam = small amount of book work + a lot of application)
Lecture 1

Theory of Algorithms and Data Structures

The Course Text
Good news! Course text available as free online e-book: http://dx.doi.org/10.1007/978-3-540-77978-0 or buy for about £20 online Covers almost all of the material we will cover in this course. Covers some material that we will not cover in this course. If you don’t understand something in lectures, read the text book and try the exercises. K. Mehlhorn and P. Sanders. If you still don’t understand something, it may Algorithms and Data be useful to have it explained in another way… Structures: The Basic Toolbox. Springer, 2008.
Lecture 1

Theory of Algorithms and Data Structures

Other Recommended Texts
More material than Mehlhorn and a different style If you can’t follow something in the course text – look it up in here Will prove very useful for this course May well be useful throughout your degree/career A good investment

T.H. Cormen et al. Introduction to Algorithms. MIT Press, 2001.
Lecture 1

Theory of Algorithms and Data Structures

Other Recommended Texts

“If you think you’re a really good programmer, read Knuth’s Art of Computer Programming. You should definitely send me a resume if you can read the whole thing.” Bill Gates

D.E. Knuth. The Art of Computer Programming. Addison Wesley, 1997. (In 3 volumes)
Lecture 1

Theory of Algorithms and Data Structures

Other Recommended Texts
Also recommended: B.R. Preiss. Data Structures and Algorithms with Object-oriented Design Patterns in Java. Wiley, 1999. (+ other Java-specific texts) R. Sedgewick. Algorithms. Addison Wesley, 1988. S.S. Skiena. The Algorithm Design Manual. Springer, 2008. Details of all texts on the webpage

Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
Muhammad ibn Mūsā al-Khwārizmī “algorism” referred to rules of performing arithmetic using Arabic numerals Evolved into “algorithm” and came to mean a definite procedure for solving a problem or performing a task “the idea behind any reasonable computer program” Something like a recipe, process, method, technique, procedure, routine etc

Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
There is no formal definition of an algorithm. The following features make a sensible informal definition: 1. Finiteness: Must terminate after a (very) finite number of steps. 2. Definiteness: Each step must be precisely defined (rigourous, unambiguous). Programming languages are designed for specifying algorithms – every statement has a definite meaning. 3. Input: Zero or more inputs. Q: Are there any useful algorithms with no inputs? 4. Output: One or more outputs. Q: Are there any useful algorithms with no outputs? 5. Effectiveness: Operations must be sufficiently basic that they can be done exactly and in a finite length of time, i.e. computable or do-able.
Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
There is no formal definition of an algorithm. The following features make a sensible informal definition: 1. Finiteness: Must terminate after a (very) finite number of steps. 2. Definiteness: Each step must be precisely defined (rigourous, unambiguous). Programming languages are designed for specifying algorithms – every statement has a definite meaning. 3. Input: Zero or more inputs. Q: Are there any useful algorithms with no inputs? 4. Output: One or more outputs. Q: Are there any useful algorithms with no outputs? 5. Effectiveness: Operations must be sufficiently basic that they can be done exactly and in a finite length of time, i.e. computable or do-able.
Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
There is no formal definition of an algorithm. The following features make a sensible informal definition: 1. Finiteness: Must terminate after a (very) finite number of steps. 2. Definiteness: Each step must be precisely defined (rigourous, unambiguous). Programming languages are designed for specifying algorithms – every statement has a definite meaning. 3. Input: Zero or more inputs. Q: Are there any useful algorithms with no inputs? 4. Output: One or more outputs. Q: Are there any useful algorithms with no outputs? 5. Effectiveness: Operations must be sufficiently basic that they can be done exactly and in a finite length of time, i.e. computable or do-able.
Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
There is no formal definition of an algorithm. The following features make a sensible informal definition: 1. Finiteness: Must terminate after a (very) finite number of steps. 2. Definiteness: Each step must be precisely defined (rigourous, unambiguous). Programming languages are designed for specifying algorithms – every statement has a definite meaning. 3. Input: Zero or more inputs. Q: Are there any useful algorithms with no inputs? 4. Output: One or more outputs. Q: Are there any useful algorithms with no outputs? 5. Effectiveness: Operations must be sufficiently basic that they can be done exactly and in a finite length of time, i.e. computable or do-able.
Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
There is no formal definition of an algorithm. The following features make a sensible informal definition: 1. Finiteness: Must terminate after a (very) finite number of steps. 2. Definiteness: Each step must be precisely defined (rigourous, unambiguous). Programming languages are designed for specifying algorithms – every statement has a definite meaning. 3. Input: Zero or more inputs. Q: Are there any useful algorithms with no inputs? 4. Output: One or more outputs. Q: Are there any useful algorithms with no outputs? 5. Effectiveness: Operations must be sufficiently basic that they can be done exactly and in a finite length of time, i.e. computable or do-able.
Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
According to our criteria, are the following valid algorithms?
Pancake batter 1. Sift the flour into a mixing bowl 2. Add pinch of salt 3. Beat the eggs lightly 4. Add to the flour and mix well in 5. Leave to rest for at least 1 hour or even overnight

Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
According to our criteria, are the following valid algorithms?
Pancake batter 1. Sift the flour into a mixing bowl 2. Add pinch of salt 3. Beat the eggs lightly 4. Add to the flour and mix well in 5. Leave to rest for at least 1 hour or even overnight Not valid (definiteness) “pinch of salt” “beat lightly” “mix well” “even overnight” All ambiguous or vague As cooks we can interpret approximately what is meant, but not precise enough for a computer to understand Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
According to our criteria, are the following valid algorithms?
Solving Chess 1. Construct the tree of all possible valid game states 2. Mark states as “won” if the player to move can win this turn or “lost” if player to move loses regardless of move chosen 3. Mark states as “lost” if all successors are marked “won” 4. Mark states as “won” if at least one successor is marked “lost” 5. Repeat 3 and 4 until no additional states are marked 6. If first move is marked “won”, then return “white always wins”

Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
According to our criteria, are the following valid algorithms?
Solving Chess 1. Construct the tree of all possible valid game states 2. Mark states as “won” if the player to move can win this turn or “lost” if player to move loses regardless of move chosen 3. Mark states as “lost” if all successors are marked “won” 4. Mark states as “won” if at least one successor is marked “lost” 5. Repeat 3 and 4 until no additional states are marked 6. If first move is marked “won”, then return “white always wins” Not valid (finiteness) ~1043 possible game states – although finite, not realistic to evaluate all Evaluate 1 million states per second = 3.2×1029 years! (Earth is about 4.5×109 years old) Lecture 1

Theory of Algorithms and Data Structures

What is an algorithm?
According to our criteria, are the following valid algorithms?
Find the greatest common divisor of positive integers m and n 1. If m 0) m and n must be positive integers Postcondition: (rjm) ^ (rjn) ^ (8i 2 N ² [(ijm) ^ (ijn) ) (i · r)]) r must divide both m and n Any other value that divides both m and n must be less than n

Lecture 1

Theory of Algorithms and Data Structures

Certifying Algorithms
Sometimes preconditions and postconditions can be easily checked Many programming languages support assertions – good practice to check pre/postconditions (more in PPS), example: Function GCD(m; n : N) : N 1: assert (m > 0) ^ (n > 0) 2: if m < n then Note: checking this 3: (m,n):=(n,m); assertion in a real 4: end if programming language 5: while n 6= 0 do is non-trivial 6: r := m mod n 7: (m; n) := (n; r) 8: end while 9: assert (ijm) ^ (ijn) ^ (8i 2 N ² [(ijm) ^ (ijn) ) (i · r)]) 10: return m
Lecture 1

Theory of Algorithms and Data Structures

Certifying Algorithms
This might seem a bit circular – isn’t checking the postcondition as hard as running the algorithm in the first place?  In many situations checking assertions can be simplified by computing additional information – a certificate  A certifying algorithm computes a certificate for the postcondition  We will see examples of algorithms later where a certificate can be computed without affecting the efficiency of the algorithm

Lecture 1

Theory of Algorithms and Data Structures

Loop Invariants
 A loop invariant is a statement which is true before and after each loop iteration  Used to help prove program correctness  Helps to clarify purpose of iterative structure  Loop variant ensures progress is made Function sum(n : N) : N 1: assert (n ¸ 0) 2: s := 0 3: for i := 1 to n do Pi¡1 4: invariant s = j=0 j 5: s := s + i 6: end while Pn 7: assert s = i=0 i 8: return s
Lecture 1

Theory of Algorithms and Data Structures

Specifications and correctness
As algorists, we need tools to:  Specify a problem so we can design algorithms to solve it  Commonly solve reduced version of problem if initially too difficult  Distinguish correct algorithms from incorrect ones  Check that the algorithm satisfies the problem specification  Compare algorithms and choose the “best” one

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
We need to know whether our algorithm will always work, i.e. produce the correct/optimal answer. Let’s look at an example: Robot Tour Optimisation Robot arm must visit a set of contact points on a circuit board to solder each point Need to choose an order in which the contact points will be visited We want to minimise time taken, i.e. shortest path Shorter tour = faster production = cheaper to produce = more profit An important algorithm!

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Robot Tour Optimisation

Input: A set P of n points Output: The shortest cycle tour that visits each point Simple idea: nearest neighbour tour Use a heuristic – pick any starting point, then next move is always to nearest point
Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Robot Tour Optimisation – Nearest Neighbour Function NearestNeighbour (P : Set of Points) 1: Pick and visit an initial point p0 2 P 2: i := 0 3: while :(P = ;) do 4: i := i + 1 5: P := P n fpi¡1 g 6: Let pi 2 P be the closest point to pi¡1 7: Visit pi 8: end while 9: Return to p0 from pi

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Nearest Neighbour Example 1

0

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Nearest Neighbour Example 1

0 1

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Nearest Neighbour Example 1

0 1 2

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Nearest Neighbour Example 1

0 7 6 2 5 4 3 1

Looks encouraging.

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Nearest Neighbour Example 1

0 7 6 2 5 4 3 1

Looks encouraging. Q: Optimal for this instance?

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Nearest Neighbour Example 1

0 7 6 2 5 4 3 1

Looks encouraging. Q: Optimal for this instance? Q: Always optimal?

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness
Nearest Neighbour Example 1

0 7 6 2 5 4 3 1

Looks encouraging. Q: Optimal for this instance? Q: Always optimal? Q: What is needed to prove incorrectness?
Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness – counter examples
Nearest Neighbour Example 2 An instance where nearest neighbour gives a suboptimal solution:

0

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness – counter examples
Nearest Neighbour Example 2 An instance where nearest neighbour gives a suboptimal solution:

0

1

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness – counter examples
Nearest Neighbour Example 2 An instance where nearest neighbour gives a suboptimal solution:

2

0

1

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness – counter examples
Nearest Neighbour Example 2 An instance where nearest neighbour gives a suboptimal solution:

2

0

1

3

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness – counter examples
Nearest Neighbour Example 2 An instance where nearest neighbour gives a suboptimal solution:

4

2

0

1

3

5

Lecture 1

Theory of Algorithms and Data Structures

Reasoning about correctness – counter examples
Nearest Neighbour Example 2 A better (optimal) solution for the same instance:

4

5

0

1

2

3

 It can be easy to prove that an algorithm is not correct – just find a counter example (the simpler, the better)  Counter example = instance of problem + optimal solution + demonstration that algorithm produces a less optimal solution  Lack of a counter example doesn’t prove it works  We will not study the formal methods to prove correctness of our algorithms  Instead, we will informally convince ourselves that they work
Lecture 1

Theory of Algorithms and Data Structures

Specifications and correctness
As algorists, we need tools to:  Specify a problem so we can design algorithms to solve it  Commonly solve reduced version of problem if initially too difficult  Distinguish correct algorithms from incorrect ones  Check that the algorithm satisfies the problem specification  Compare algorithms and choose the “best” one  There may be many alternative algorithms that satisfy a specification  We want the one that is “best” for our purposes  Often, best = most efficient. Is this always the case?  The subject of much of this course!
Lecture 1

Theory of Algorithms and Data Structures

Analysis of Algorithms
The theoretical study of computer-program performance and resource usage. Is performance the most important measure of an algorithm? What’s more important than performance?

Lecture 1

Theory of Algorithms and Data Structures

Analysis of Algorithms
The theoretical study of computer-program performance and resource usage. Is performance the most important measure of an algorithm? What’s more important than performance?  correctness  security  user-friendliness  maintainability  robustness/reliability  functionality/features  simplicity  programmer time  modularity  extensibility

Performance is the currency of computation We can use it to buy other things: functionality, user-friendliness etc
Lecture 1

Theory of Algorithms and Data Structures

Analysis of Algorithms
Why study analysis of algorithms? A few cliches:

 Some algorithms experts are lousy programmers, but no good programmer can be bad at algorithms  Algorithm design and analysis is the essence of what we do as computer scientists  With more sophisticated software engineering systems, the demand for mundane programmers will diminish  Soon all known algorithms will be available in libraries

Lecture 1

Theory of Algorithms and Data Structures

Analysis of Algorithms
Why study analysis of algorithms? A few cliches:

 Great thinkers will always be needed: “I can develop a new algorithm for you”  “To be a good programmer you can either study programming for 10 years or take an algorithms course and study programming for 2 years”  Making things faster is fun! Some people try to design faster cars, we try to design faster algorithms!

Lecture 1

Theory of Algorithms and Data Structures

Conclusions
We should now know:  What an algorithm is  How to specify a problem  How to show algorithm incorrectness by counter example  Some justification for why algorithm analysis is important Recommended reading for this lecture:  Mehlhorn Chapter 1 & 2.3/2.4  Knuth Vol. 1, Section 1.1  Cormen Chapter 1

Lecture 1