Algorithms and Data Structures - Course Introduction
Transcript of Algorithms and Data Structures - Course Introduction
Example
A sequence of numbers
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .
c© 2006 Antonio Carzaniga
Example
A sequence of numbers
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .
The well-known Fibonacci sequence
Leonardo da Pisa (ca. 1170–ca. 1250)son of Guglielmo “Bonaccio”a.k.a. Leonardo Fibonacci
c© 2006 Antonio Carzaniga
The Fibonacci Sequence
Mathematical definition of the Fibonacci sequence
Fn =
0 if n = 0
1 if n = 1
Fn−1 + Fn−2 if n > 1
c© 2006 Antonio Carzaniga
The Fibonacci Sequence
Mathematical definition of the Fibonacci sequence
Fn =
0 if n = 0
1 if n = 1
Fn−1 + Fn−2 if n > 1
In “pseudo-code”
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
c© 2006 Antonio Carzaniga
Questions on Our First Algorithm
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
c© 2006 Antonio Carzaniga
Questions on Our First Algorithm
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
1. Is the algorithm correct?◮ for every valid input, does it terminate?◮ if so, does it do the right thing?
c© 2006 Antonio Carzaniga
Questions on Our First Algorithm
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
1. Is the algorithm correct?◮ for every valid input, does it terminate?◮ if so, does it do the right thing?
2. How much time does it take to complete?
c© 2006 Antonio Carzaniga
Questions on Our First Algorithm
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
1. Is the algorithm correct?◮ for every valid input, does it terminate?◮ if so, does it do the right thing?
2. How much time does it take to complete?
3. Can we do better?c© 2006 Antonio Carzaniga
Correctness
c© 2006 Antonio Carzaniga
Correctness
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
Fn =
0 if n = 0
1 if n = 1
Fn−1 + Fn−2 if n > 1
c© 2006 Antonio Carzaniga
Correctness
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
Fn =
0 if n = 0
1 if n = 1
Fn−1 + Fn−2 if n > 1
The algorithm is clearly correct◮ assuming n ≥ 0
c© 2006 Antonio Carzaniga
Performance
How long does it take?
c© 2006 Antonio Carzaniga
Performance
How long does it take?
Let’s try it. . .
c© 2006 Antonio Carzaniga
Results
0
5
10
15
20
25
30
20 25 30 35 40 45
Tim
e (s
econ
ds)
n
"Python""Scheme"
"C""C-wiz""Java"
"C-gcc"
c© 2006 Antonio Carzaniga
Comments
c© 2006 Antonio Carzaniga
Comments
Different implementations perform differently
◮ it is better to let the compiler do the optimization
◮ simple language tricks don’t seem to pay off
c© 2006 Antonio Carzaniga
Comments
Different implementations perform differently
◮ it is better to let the compiler do the optimization
◮ simple language tricks don’t seem to pay off
However, the differences do not seem to be substantial
◮ all implementations, sooner or later, seem to hit a wall. . .
c© 2006 Antonio Carzaniga
Comments
Different implementations perform differently
◮ it is better to let the compiler do the optimization
◮ simple language tricks don’t seem to pay off
However, the differences do not seem to be substantial
◮ all implementations, sooner or later, seem to hit a wall. . .
Conclusion: the problem lies with the algorithm
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm
We need a mathematical characterization of the performance ofthe algorithm
We’ll call it the algorithm’s computational complexity
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm
We need a mathematical characterization of the performance ofthe algorithm
We’ll call it the algorithm’s computational complexity
Formally, but a bit superficially for now, let T (n) be the number ofbasic steps needed to compute FIBONACCI(n)
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm
We need a mathematical characterization of the performance ofthe algorithm
We’ll call it the algorithm’s computational complexity
Formally, but a bit superficially for now, let T (n) be the number ofbasic steps needed to compute FIBONACCI(n)
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm
We need a mathematical characterization of the performance ofthe algorithm
We’ll call it the algorithm’s computational complexity
Formally, but a bit superficially for now, let T (n) be the number ofbasic steps needed to compute FIBONACCI(n)
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
T (0) = 2; T (1) = 3
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm
We need a mathematical characterization of the performance ofthe algorithm
We’ll call it the algorithm’s computational complexity
Formally, but a bit superficially for now, let T (n) be the number ofbasic steps needed to compute FIBONACCI(n)
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
T (0) = 2; T (1) = 3
T (n) = T (n− 1) + T (n− 2) + 3
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm
We need a mathematical characterization of the performance ofthe algorithm
We’ll call it the algorithm’s computational complexity
Formally, but a bit superficially for now, let T (n) be the number ofbasic steps needed to compute FIBONACCI(n)
FIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)
T (0) = 2; T (1) = 3
T (n) = T (n− 1) + T (n− 2) + 3 ⇒ T (n) ≥ Fn
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .
Fn ≥ 2Fn−2
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .
Fn ≥ 2Fn−2 ≥ 2(2Fn−4)
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .
Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6))
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .
Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . .
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .
Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . . ≥ 2n2
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .
Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . . ≥ 2n2
This means that
T (n) ≥ (√
2)n ≈ (1.4)n
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .
Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . . ≥ 2n2
This means that
T (n) ≥ (√
2)n ≈ (1.4)n
T (n) grows exponentially with n
c© 2006 Antonio Carzaniga
Complexity of Our First Algorithm (2)
So, let’s try to understand how Fn grows with n
T (n) ≥ Fn = Fn−1 + Fn−2
Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .
Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . . ≥ 2n2
This means that
T (n) ≥ (√
2)n ≈ (1.4)n
T (n) grows exponentially with n
Can we do better?
c© 2006 Antonio Carzaniga
A Better Algorithm
Again, the sequence is 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .
c© 2006 Antonio Carzaniga
A Better Algorithm
Again, the sequence is 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .
Idea: we can build Fn from the ground up!
c© 2006 Antonio Carzaniga
A Better Algorithm
Again, the sequence is 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .
Idea: we can build Fn from the ground up!
SMARTFIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else pprev ← 06 prev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev
10 prev ← f11 return f
c© 2006 Antonio Carzaniga
Results
0
5
10
15
20
25
30
0 20 40 60 80 100
Tim
e (s
econ
ds)
n
"PythonSmart""Scheme"
"C""C-wiz""Java"
"C-gcc"
c© 2006 Antonio Carzaniga
Complexity of SmartFibonacci
SMARTFIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else prev ← 06 pprev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev
10 prev ← f11 return f
c© 2006 Antonio Carzaniga
Complexity of SmartFibonacci
SMARTFIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else prev ← 06 pprev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev
10 prev ← f11 return f
T (n) =
c© 2006 Antonio Carzaniga
Complexity of SmartFibonacci
SMARTFIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else prev ← 06 pprev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev
10 prev ← f11 return f
T (n) = 6 + 6(n− 1)
c© 2006 Antonio Carzaniga
Complexity of SmartFibonacci
SMARTFIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else prev ← 06 pprev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev
10 prev ← f11 return f
T (n) = 6 + 6(n− 1) = 6n
c© 2006 Antonio Carzaniga
Complexity of SmartFibonacci
SMARTFIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else prev ← 06 pprev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev
10 prev ← f11 return f
T (n) = 6 + 6(n− 1) = 6n
The complexity of SMARTFIBONACCI(n) is linear in n
c© 2006 Antonio Carzaniga
Complexity of SmartFibonacci
SMARTFIBONACCI(n)
1 if n = 02 then return 03 elseif n = 14 then return 15 else prev ← 06 pprev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev
10 prev ← f11 return f
T (n) = 6 + 6(n− 1) = 6n
The complexity of SMARTFIBONACCI(n) is linear in n . . . almost. . .
c© 2006 Antonio Carzaniga