Introduction to Asymptotic Notations.

Before implementing any algorithm as a program, we need to analyze which of the algorithm is good in terms of time and space.

Time: Which one can execute faster.
Memory: Which one can take less memory.  

Here, we will learn how to design and analyze the best algorithm in terms of time and space and how asymptotic notations are used to measure the complexity of the algorithm.

What are Asymptotic Notations?

Asymptotic notations are the mathematical tools to measure the space and time complexity of an algorithm. There are manly three asymptotic notations is use to show the time complexity of the algorithm.

O(n) Big O notation. (Worst Case)

Big O notation is the most used notation to measure the performance of any algorithm by defining its order of growth.

Big O notation

Assume that the time and input are going to increase like in the above graph then what is the worst-case or upper bound for this function?
We have to find out another function in such a way that the function is greater than the given f(n), let's call it c.g(n), after some limit n=no


For any given algorithm, we should be able to find out what is its time complexity or what its f(n) in terms of input n. Now after finding out that, if we can bound that function by some another function say c.g(n) in such a way that after some input no, the value of c.g(n) is always greater than f(n).

[ f(n)<=c.g(n) ], after some n where is n>=no
c, n are real numbers, c > 0 and n >=no 

Definition: f(n)  O(g(n)) if there exists constants c > 0 and no > =1 such that 0 <= f(n) <= c.g(n) for all n >= no

Example: Is f(n) is big O of g(n) [f(n) = O(g(n))] where f(n) = 3n+2 and g(n) = n. 

Solution: At first we have to find two constants c and no such that, 
f(n)<=c.g(n), c > 0 and n >=no
3n+2 < cn. 
let's take c=4
3n+2 < 4n
=> n>=2
Therefore, f(n) = O(g(n)). 
Therefore, g(n) = n bounds the function f(n) which means f(n) is upper bounded by g(n).

So if n is bounding this f(n) then definitely n^2 will bound. It means if g(n) = n is satisfying then for any value greater than n (n^2, n^3,----n^n) will also bound 3n+2, but we need to keep in mind that big O is always upper bound, so all the answers are correct but we will always go for least bound or tightest bound and n is the tightest bound for given f(n). 

Ω(n) Omega notation. (Best Case)

Omega notation gives us a lower bound of the complexity in the best case analysis. In this notation, we analyze the minimum number of steps that need to be executed. For example, In Linear search, the best case is when your search element is present in the first place.
Omega

We have to find out another function in such a way that the function is smaller than the given f(n), let's call it c.g(n), after some limit n=no
Now after finding out that, if we can bound that function by some another function say c.g(n) in such a way that after some input no, the value of c.g(n) is always smaller than f(n).

[ f(n)>=c.g(n) ], after some n where is n>=no
c, n are real numbers, c > 0 and n >=no 

Definition: f (n) ∈ Ω(g(n)) if there exists constants c > 0 and n0 >= 1 such that 0 ≤ c ·g(n) ≤ f (n) for all n ≥ n0

Example: Is f(n) is big Omega of g(n) where f(n) = 3n+2 and g(n) = n.

Solution: 
3n+2 >= cn. is true even for c = 1 
Therefore, the condition is satisfied and 3n+2 is lower bounded by n. 
The given function f(n) can be lower bounded by any value lower than n but it's better to take the closest bound. 

Θ(n) Theta notation. (Average Case)

Theta notation gives us a tight bound of the complexity in the average case analysis. In this notation, we calculate the average by considering different types of inputs including the best case and the worst case.
Theta

Here we have to find out both the upper bound and lower bound, by a function by a function by varying c.
If f(n) is bounded by c1g(n) and c2g(n) then, f(n) is Θg(n).

Definition: f (n) ∈ Θ(g(n)) if there exists constants c1 > 0, c2 > 0 and n0 > 1 such that c1 ·g(n) ≤ f (n) ≤ c2 ·g(n) for all n ≥ n0

Example: Is f(n) is big Theta of g(n) where f(n) = 3n+2 and g(n) = n.

Solution: 
f(n)  ≤  c2 ·g(n)
3n+2 ≤ 4n, true for n0 ≥ 1.

f(n) ≥ c1 ·g(n)
3n+2 ≥ n, true for n0 ≥ 1 and c = 1

This means, f(n) = 3n+2 is bounded by g(n) from both upper as well as lower bound, for upper bound  c2 = 4, and for lower bound c1 = 1, both the cases are valid for n0 ≥ 1.

Note: Sometimes, Θ is also called asymptotically equal which means if the leading term in the function is going to be 1, then we can take n as g(n) because they are asymptotically equal. 
Ex: f(n) = 3n^2+n+1 = Θ(n^2). 
f(n) = 5n^3+n+1 = Θ(n^3). 

Related post

If you like this post and find it useful then you can show your support by donating a small amount from your heart. You can also show your support by following us on Facebook and Twitter. 

Post a Comment

Previous Post Next Post