Why bubble sort is faster than quick sort
Question:
I try to sort my list using two algorithms of sort: bubble and quick.
For this purpose i use algorithms module and bubble_sort , quick_sort respectively. As I know complexity of first algorithm is n^2 and second is n*log(n). But i get unexpected output from this code:
from algorithms.sorting import bubble_sort, quick_sort
import time
my_list = [1, 12, 33, 14, 52, 16, 71, 18, 94]
start1 = time.time()
for i in range(1000000):
bubble_sort.sort(my_list)
end1 = time.time()
start2 = time.time()
for i in range(1000000):
quick_sort.sort(my_list)
end2 = time.time()
print('Bubble sort:', end1 - start1)
print('Quick sort:',end2 - start2)
Output:
>>> Bubble sort: 7.04760217666626
>>> Quick sort: 8.181402921676636
Why in this case bubble sort is faster than quick sort?
Answers:
The time complexity of an algorithm does not give any guarantees about the runtime; instead, it gives an estimate for asymptotic behavior of that algorithm. In your case, n = 9 very small, so the effects of hidden constants in the algorithms will become more important than the differences in the time complexities themselves.
Try rerunning your program, but this time with a much larger value of n (say n=10000). To test the general behavior of both algorithms, make sure that your input list is randomly ordered. You could also experiment with edge-case lists (i.e. already sorted) to observe the worst-case performance of quicksort and the best-case performance of bubble sort.
The worst case running time of quick sort is O(n^2). The worst case depends on pivot selection strategy, usually it occurs for a already sorted array (which your array is).
Also, for small data set, bubble sort or other simple sorting algorithm usually works faster than more complex algorithms. The reason is, for each iteration, simple algorithms does less calculation than complex algorithms.
For example, say bubble sort takes 3ms per iteration while quicksort takes 20ms. So for an array with 10 items.
In this case bubble sort takes 10*10*3 = 300ms.
And quicksort takes 10*log2(10)*20 = 664ms. (Considering the average case)
So bubble sort is faster here. But as we take larger dataset, quicksort becomes increasingly efficient due to lower run-time complexity.
So what are the worst-case run-times here?
Quicksort: n^2 and
Bubblesort: n^2
Remember that worst case is not always a good indicator of real world performance. In the average case,
Quicksort: nlog(n)
Bubblesort: n^2
So based on this, Quicksort is faster than Bubblesort.
However, Quicksort handles degenerate cases poorly. When the list is in almost-sorted order already, Quicksort is going to keep recursing. Bubblesort will stop as soon as its done, making Bubblesort faster in such cases.
Firstly, try the sort on a much bigger array, so that the quadratic complexity could take precedence over the logarithmic complexity.
Note: regarding the logarithmic complexity, be careful that the log(n) as far as quicksort is concerned, is not log10, it’s log2 -> O(n) should be denoted as n * lg(n)
Secondly, there is no reason to sort an already sorted array… try this:
import numpy as np
arr = np.linspace(-1e3, 1e3, 1e5)
np.random.shuffle(arr) # Shuffling array preserving the content
If you algorithm does not accept numpy array, convert it to list:
arr_l = list(arr)
Mathematically n^2 is greater than nlog(n) for all n >= 1.
So bubble sort{O(n^2)} should be slower than quick sort{O(nlog n)} for n = 9 (from example).
But the actual complexity is:
Big-O Bubble Sort: n(n-1) which is equivalent O(n^2)
Big-O Quick Sort: O(n(log n))
But as n=9 is very small, n^2 and n are comparable and the assumption n^2-n equivalent to n becomes wrong.
As for proof:
n^2-n for n=9 is 7.2
n(log n) for n=9 is 8.5
which is the same as mention in the question.
The reason bubble performs better than quicksort in some cases is lost to many Computer Scientists: It’s true that bubble sort will result in many more "element swaps" than with quicksort. But there is more to performance than eliminating swaps. Bubblesort is generally an "in-line" routine (as opposed to a "forced" function call). Even if it’s coded as a function, a good optimizing compiler will compile it in-line. And if it is not, there is still only one function call per sort.
Quicksort however, relies in recursion, which forces a function call mechanism. Every time a recursion cycle occurs (a call to quicksort from within quicksort), the entire environment needs to be saved on the stack. Then a transfer of control is executed. At the end of the recursion, the entire environment needs to be restored, and another transfer of control is executed (return to the calling function). There can be a very heavy performance penalty in frequent recursions. In my humble opinion, many look at the "elegance" of quicksort and "recursion" in general, but overlook its overhead.
Not saying this in a vacuum, I wrote some benchmarks and bubblesort swaps a lot more, but still beats quicksort.
In those benchmarks, I found that even with data that is already "in order" quicksort will recurse N-1 times, with N being the number of elements to sort.
Notation Big O does not provide an specific runtime, but an estimation of it. Nevertheless, it is required that those estimations might variate regarding the current order of your array.
When comparing both algorithms, meaning Bubble sort and Arrays.sort, we need to consider the following:
Arrays.sort() runs in O(n log (n)) as best, average or performance.- Bubble sort, runs in:
- List item
O(n ^ 2) as worst and average case, but O(n) on its best
Therefore, the array which is being sorted, might be working on the best performance for Bubble sort, ergo will be faster
I try to sort my list using two algorithms of sort: bubble and quick.
For this purpose i use algorithms module and bubble_sort , quick_sort respectively. As I know complexity of first algorithm is n^2 and second is n*log(n). But i get unexpected output from this code:
from algorithms.sorting import bubble_sort, quick_sort
import time
my_list = [1, 12, 33, 14, 52, 16, 71, 18, 94]
start1 = time.time()
for i in range(1000000):
bubble_sort.sort(my_list)
end1 = time.time()
start2 = time.time()
for i in range(1000000):
quick_sort.sort(my_list)
end2 = time.time()
print('Bubble sort:', end1 - start1)
print('Quick sort:',end2 - start2)
Output:
>>> Bubble sort: 7.04760217666626
>>> Quick sort: 8.181402921676636
Why in this case bubble sort is faster than quick sort?
The time complexity of an algorithm does not give any guarantees about the runtime; instead, it gives an estimate for asymptotic behavior of that algorithm. In your case, n = 9 very small, so the effects of hidden constants in the algorithms will become more important than the differences in the time complexities themselves.
Try rerunning your program, but this time with a much larger value of n (say n=10000). To test the general behavior of both algorithms, make sure that your input list is randomly ordered. You could also experiment with edge-case lists (i.e. already sorted) to observe the worst-case performance of quicksort and the best-case performance of bubble sort.
The worst case running time of quick sort is O(n^2). The worst case depends on pivot selection strategy, usually it occurs for a already sorted array (which your array is).
Also, for small data set, bubble sort or other simple sorting algorithm usually works faster than more complex algorithms. The reason is, for each iteration, simple algorithms does less calculation than complex algorithms.
For example, say bubble sort takes 3ms per iteration while quicksort takes 20ms. So for an array with 10 items.
In this case bubble sort takes 10*10*3 = 300ms.
And quicksort takes 10*log2(10)*20 = 664ms. (Considering the average case)
So bubble sort is faster here. But as we take larger dataset, quicksort becomes increasingly efficient due to lower run-time complexity.
So what are the worst-case run-times here?
Quicksort: n^2 and
Bubblesort: n^2
Remember that worst case is not always a good indicator of real world performance. In the average case,
Quicksort: nlog(n)
Bubblesort: n^2
So based on this, Quicksort is faster than Bubblesort.
However, Quicksort handles degenerate cases poorly. When the list is in almost-sorted order already, Quicksort is going to keep recursing. Bubblesort will stop as soon as its done, making Bubblesort faster in such cases.
Firstly, try the sort on a much bigger array, so that the quadratic complexity could take precedence over the logarithmic complexity.
Note: regarding the logarithmic complexity, be careful that the log(n) as far as quicksort is concerned, is not log10, it’s log2 -> O(n) should be denoted as n * lg(n)
Secondly, there is no reason to sort an already sorted array… try this:
import numpy as np
arr = np.linspace(-1e3, 1e3, 1e5)
np.random.shuffle(arr) # Shuffling array preserving the content
If you algorithm does not accept numpy array, convert it to list:
arr_l = list(arr)
Mathematically n^2 is greater than nlog(n) for all n >= 1.
So bubble sort{O(n^2)} should be slower than quick sort{O(nlog n)} for n = 9 (from example).
But the actual complexity is:
Big-O Bubble Sort: n(n-1) which is equivalent O(n^2)
Big-O Quick Sort: O(n(log n))
But as n=9 is very small, n^2 and n are comparable and the assumption n^2-n equivalent to n becomes wrong.
As for proof:
n^2-n for n=9 is 7.2
n(log n) for n=9 is 8.5
which is the same as mention in the question.
The reason bubble performs better than quicksort in some cases is lost to many Computer Scientists: It’s true that bubble sort will result in many more "element swaps" than with quicksort. But there is more to performance than eliminating swaps. Bubblesort is generally an "in-line" routine (as opposed to a "forced" function call). Even if it’s coded as a function, a good optimizing compiler will compile it in-line. And if it is not, there is still only one function call per sort.
Quicksort however, relies in recursion, which forces a function call mechanism. Every time a recursion cycle occurs (a call to quicksort from within quicksort), the entire environment needs to be saved on the stack. Then a transfer of control is executed. At the end of the recursion, the entire environment needs to be restored, and another transfer of control is executed (return to the calling function). There can be a very heavy performance penalty in frequent recursions. In my humble opinion, many look at the "elegance" of quicksort and "recursion" in general, but overlook its overhead.
Not saying this in a vacuum, I wrote some benchmarks and bubblesort swaps a lot more, but still beats quicksort.
In those benchmarks, I found that even with data that is already "in order" quicksort will recurse N-1 times, with N being the number of elements to sort.
Notation Big O does not provide an specific runtime, but an estimation of it. Nevertheless, it is required that those estimations might variate regarding the current order of your array.
When comparing both algorithms, meaning Bubble sort and Arrays.sort, we need to consider the following:
Arrays.sort()runs inO(n log (n))as best, average or performance.- Bubble sort, runs in:
- List item
O(n ^ 2)as worst and average case, butO(n)on its best
Therefore, the array which is being sorted, might be working on the best performance for Bubble sort, ergo will be faster