# data[].attributes.attachment_link.title
Shortest Path Algorithms Explained (Dijkstra's & Bellman-Ford)

# data[].attributes.attachment_link.title
Algoritmos BFS y DFS (Recorridos en Grafos)

# data[].attributes.attachment_link.readable_text
With Heapsort, every Java developer first thinks of the Java heap. This article will show you that Heapsort is something completely different - and how Heapsort works precisely.

You'll find out in detail:

What is a Heap?

How does the Heapsort algorithm work?

What does the Heapsort source code look like?

How to determine Heapsort's time complexity?

What is Bottom-up Heapsort, and what are its advantages?

How does Heapsort compare to Quicksort and Merge Sort?

A "heap" is a binary tree in which each node is either greater than or equal to its children ("max heap") - or less than or equal to its children ("min heap").

Here is a simple example of a "max heap":

The 9 is greater than the 8 and the 5; the 8 is greater than the 7 and the 2; etc.

A heap is projected onto an array by transferring its elements line by line from top left to bottom right into the array:

The heap shown above looks like this as an array:

In a "max heap", the largest element is always at the top - in the array form, it is, therefore, on the far left. The following section explains how to use this characteristic for sorting.

The heapsort algorithm consists of two phases: In the first phase, the array to be sorted is converted into a max heap. And in the second phase, the largest element (i.e., the one at the tree root) is removed, and a new max heap is created from the remaining elements.

The following sections explain the two phases in detail using an example:

Phase 1: Creating the Heap

The array to be sorted must first be converted into a heap. For this purpose, no new data structure is created, but the numbers are rearranged within the array so that the heap structure described above is created.

In the following example, I explain how exactly this is done using the number sequence known from the previous parts of the article series: [3, 7, 1, 8, 2, 5, 9, 4, 6].

We "project" these numbers onto a binary tree, as described above. The binary tree is not a separate data structure, but only a thought construct - in the computer's memory, the elements are located exclusively in the array.

This tree does not yet represent a max heap. The definition of a max heap is that parents are always greater than or equal to their children.

To create a max heap, we now visit all parent nodes - backward from the last one to the first - and make sure that the heap condition for the respective node and the one below is fulfilled. We do this using the so-called heapify() method.

Invocation No. 1 of the Heapify Method

The heapify() method is called first for the last parent node. Parent nodes are 3, 7, 1, and 8. The last parent node is 8. The heapify() function checks if the children are smaller than the parent node. 4 and 6 are smaller than 8, so at this parent node, the heap condition is fulfilled, and the heapify() function is finished.

Invocation No. 2 of the Heapify Method

Second, heapify() is called for the penultimate node: the 1. Its children 5 and 9 are both greater than 1, so the heap condition is violated. To restore the heap condition, we now swap the larger child with the parent node, i.e., the 9 with the 1. The heapify() method is now finished again.

Invocation No. 3 of the Heapify Method

Now heapify() is called on the 7. Child nodes are 8 and 2; only the 8 is larger than the parent node. So we exchange the 7 with the 8:

Since the child node we just swapped has two children itself, the heapify() method must now check if the heap condition for this child node is still valid. In this case, the 7 is greater than 4 and 6; the heap condition is fulfilled, and the heapify() function is finished.

Invocation No. 4 of the Heapify Method

Now we have arrived at the root node with element 3. Both child nodes, 8 and 9 are larger, while 9 is the largest child and is, therefore, swapped with the parent node:

Again, the swapped child node has children itself, so we need to check the heap condition on this child node. The 5 is greater than the 3, i.e., the heap condition is not fulfilled. It must be restored by swapping the 5 and the 3:

The fourth and last call of the heapify() function has finished. A max heap has been created:

Which brings us to phase two of the heapsort algorithm.

Phase 2: Sorting the Array

In phase 2, we take advantage of the fact that the largest element of the max heap is always at its root (in the array: on the far left).

Phase 2, Step 1: Swapping the Root and Last Elements

The root element (the 9) is now swapped with the last element (the 6) so that the 9 is at its final position at the end of the array (marked blue in the array). We also remove this element from the tree (displayed in grey):

After we've placed the 6 at the root of the tree, it is no longer a max heap. Therefore, in the next step, we will "repair" the heap.

Phase 2, Step 2: Restoring the Heap Condition

To restore the heap condition, we call the heapify() method known from phase 1 on the root node. This means we compare the 6 with its children, 8 and 5; the 8 is bigger, so we swap it with the 6:

The swapped child node has, in turn, two children, the 7 and the 2. The 7 is larger than the 6, and we swap these two elements as well:

The exchanged child node also has a child, the 4. The 6 is greater than the 4, so the heap condition is fulfilled at this node. The heapify() function is finished, and we have a max heap again:

The largest number of the remaining array, 8, is now in the first position. We swap it with the last element of the tree. Since we have shortened the tree by one element, the last element of the tree is on the second last field of the array:

Now, the last two fields of the array are sorted.

At the root, the heap condition is violated again. We repair the tree by calling heapify() on the root element (the following picture shows all heapify steps at once).

We repeat the process until there is only one element left in the tree:

This element is the smallest and remains at the beginning of the array. The algorithm is finished, the array is sorted:

Heapsort Java Code Example

In this section, you'll find the source code of Heapsort.

The sort() method first calls buildHeap() to initially build the heap.

In the following loop, the variable swapToPos iterates backward from the end of the array to its second field. In the loop body, the first element is swapped with the one at the swapToPos position, and then the heapify() method is called on the subarray up to (exclusive) the swapToPos position:

public class HeapSort { public void sort(int[] elements) { buildHeap(elements); for (int swapToPos = elements.length - 1; swapToPos > 0; swapToPos--) { ArrayUtils.swap(elements, 0, swapToPos); heapify(elements, swapToPos, 0); } } [...]

The buildHeap() method calls heapify() for each parent node, starting with the last one, and passes to this method the array, the length of the subarray representing the heap, and the position of the parent node where heapify() should start:

The heapify() method checks whether a child node is larger than the parent node. If this is the case, the parent element is swapped with the larger child element, and the process is repeated on the child node.

(You could also work with recursion here, but this would have a negative effect on the space complexity)

{ while (true) { int leftChildPos = parentPos * 2 + 1; int rightChildPos = parentPos * 2 + 2; int largestPos = parentPos; if (leftChildPos < length && heap[leftChildPos] > heap[largestPos]) { largestPos = leftChildPos; } if (rightChildPos < length && heap[rightChildPos] > heap[largestPos]) { largestPos = rightChildPos; } if (largestPos == parentPos) { break; } ArrayUtils.swap(heap, parentPos, largestPos); parentPos = largestPos; } }

You can find the source code in the HeapSort class in the GitHub repository. It is slightly different from the class printed here: The class in the repository implements the SortAlgorithm interface to be interchangeable within the test framework.

Heapsort Time Complexity

Click on the following link for an introduction to "time complexity" and "O notation" (with examples and diagrams).

Time Complexity of the heapify() Method

Let's start with the heapify() method since we also need it for the heap's initial build.

In the heapify() function, we walk through the tree from top to bottom. The height of a binary tree (the root not being counted) of size n is at most, i.e., if the number of elements doubles, the tree becomes only one level deeper:

The complexity for the heapify() function is accordingly O(log n).

Time Complexity of the buildHeap() Method

To initially build the heap, the heapify() method is called for each parent node - backward, starting with the last node and ending at the tree root.

A heap of size n has n/2 (rounded down) parent nodes:

Since the complexity of the heapify() method is O(log n) as shown above, the complexity for the buildHeap() method is, therefore, maximum* O(n log n).

Total Time Complexity of Heapsort

The heapify() method is called n-1 times. So the total complexity for repairing the heap is also O(n log n).

Both sub-algorithms, therefore, have the same time complexity. Hence:

The time complexity of Heapsort is: O(n log n)

Time Complexity for Building the Heap - In-Depth Analysis

This section is very mathematical and not necessary for determining the time complexity of the overall algorithm (which we have already completed). You could, therefore, skip this section.

We have seen above that the buildHeap() method calls heapify() for each parent node. What we have not considered so far is that the depth of the subtrees, on which heapify() is called, varies. The following graphic illustrates this ( d stands for the depth of the subtrees)

The heapify() method is called at most for n/4 trees of depth 1, for n/8 trees of depth 2, for n/16 trees of depth 3, etc.

The maximum number of swap operations in the heapify() method is equal to the depth of the subtree on which it is called.

The maximum number of swap operations is therefore:
Smax = n/4 · 1 + n/8 · 2 + n/16 · 3 + n/32 · 4 + ...
If we multiply both sides of the term by 2, we get:
2 · Smax = n/2 · 1 + n/4 · 2 + n/8 · 3 + n/16 · 4 + ...
Let us place both terms on top of each other:

Both terms contain n/4, n/8, n/16 etc. with a factor differing by the constant 1. If we subtract the terms, we get:
2 · Smax - Smax = n/2 · 1 + n/4 · (2 - 1) + n/8 · (3 - 2) + n/16 · (4 - 3) + ...
This can be simplified:

Or:

The term 1/2 + 1/4 + 1/8 + 1/16 + ... approaches 1, as shown in the following diagram:

Thus the formula can finally be simplified to:

We have thus shown that the effort required to build the heap is linear, i.e., the time complexity is O(n).

However, the total complexity of O(n log n) mentioned above does not change due to the lower complexity class of a partial algorithm.

Runtime of the Java Heapsort Example

The UltimateSort class can be used to determine the runtime of different sorting algorithms for different input sizes.

The following table shows the medians of the runtimes for sorting randomly arranged, as well as ascending and descending presorted elements, after 50 repetitions (this is only an excerpt for the sake of clarity; the complete result can be found here):

Here are the complete measurements as a diagram:

You can see clearly:

When doubling the input quantity, sorting takes a little more than twice as long; this corresponds to the expected quasilinear runtime O(n log n).

For presorted input data, Heapsort is about three times faster than for unsorted data.

Input data sorted in ascending order will be sorted about as fast as input data sorted in descending order.

Why Is Heapsort Faster for Presorted Input Data?

To address this question, I use the program CountOperations to measure the number of compare, read, and write operations of Heapsort for unsorted, ascending, and descending sorted data for the respective phases.

You can find the result in the file CountOperations_Heapsort.log. The results of the test are:

If the input data is sorted in descending order, there are only about half as many comparisons in phase 1 as there are for unsorted or ascending data; there are also no swap operations. This is because a descending sorted array already corresponds to a max heap.

Input data sorted in ascending order correspond to a min heap. The tree must be completely reversed in the buildHeap() phase, so in this case, we have about a third more swap operations than with randomly arranged data, in which the heap condition is already fulfilled on some subtrees.

In phase 2, the number of operations differs only slightly.

Then how can we explain that heapsort is about three times faster for both ascending and descending presorted input data?

We find the answer in the so-called branch prediction.

With presorted input data, the comparison operations always lead to the same result. If the branch prediction now assumes that the comparisons will also lead to the same result in the future, the CPU's instruction pipelines can be fully utilized.

With unsorted input data, however, no reliable statement can be made about future comparison results. As a result, the instruction pipeline must often be deleted and refilled.

Bottom-up Heapsort is a variant in which the heapify() method makes do with fewer comparisons through smart optimization. This is advantageous if, for example, we don't compare int primitives, but objects with a time-consuming compareTo() function.

In the regular heapify(), we perform two comparisons on each node from top to bottom to find the largest of three elements:

1. Parent node with left child

2. The larger node from the first comparison with the second child

Bottom-Up Heapsort Algorithm

Bottom-up Heapsort, on the other hand, only compares the two children and follows the larger child to the end of the tree ("top-down"). From there, the algorithm goes back towards the tree root ("bottom-up") and searches for the first element larger than the root. From this position, all elements are moved one position towards the root, and the root element is placed in the field that has become free.

The following example should make it easier to understand.

Bottom-Up Heapsort Example

In the following example, we compare the 9 and 4, then the children of the 9 - the 8 and the 6, and finally the children of the 8 - the 7 and the 3:

In this way, we reach the 7 and compare it with the tree root, the 5:

The 5 is smaller than the 7, which means that the root element must be passed all the way down:

In the end, this leads to the same result as the regular heapify().

Bottom-up Heapsort takes advantage of the fact that the root element is usually shifted very far down. The reason is that it comes from the end of the tree after each iteration and is therefore relatively small.

This means that fewer comparisons are necessary if one comparison per node is made all the way down and then a short distance up again - compared to two comparisons per node from top to bottom:

Bottom-Up Heapsort Source Code

The class BottomUpHeapsort inherits from Heapsort and overwrites its heapify() method with the following:

 { int leafPos = findLeaf(heap, length, rootPos); int nodePos = findTargetNodeBottomUp(heap, rootPos, leafPos); if (rootPos == nodePos) return; int nodeValue = heap[nodePos]; heap[nodePos] = heap[rootPos]; while (nodePos > rootPos) { int parentPos = getParentPos(nodePos); int parentValue = heap[parentPos]; heap[parentPos] = nodeValue; nodePos = getParentPos(nodePos); nodeValue = parentValue; } }

The findLeaf() method compares two children and follows the larger one until the end of the tree is reached (or a node with only one child):

{ int pos = rootPos; int leftChildPos = pos * 2 + 1; int rightChildPos = pos * 2 + 2; while (rightChildPos < length) { if (heap[rightChildPos] > heap[leftChildPos]) { pos = rightChildPos; } else { pos = leftChildPos; } leftChildPos = pos * 2 + 1; rightChildPos = pos * 2 + 2; } if (leftChildPos < length) { pos = leftChildPos; } return pos; }

The method findTargetNodeBottomUp() searches from bottom to top for the first element that is not smaller than the root node:

int findTargetNodeBottomUp(int[] heap, int rootPos, int leafPos) { int parent = heap[rootPos]; while (leafPos != rootPos && heap[leafPos] < parent) { leafPos = getParentPos(leafPos); } return leafPos; }

And finally the getParentPos() method:

Bottom-Up Heapsort Performance

We can also measure the performance of Bottom-Up Heapsort with UltimateTest. You can find the results in UltimateTest_Heapsort.log. The following diagram shows the runtimes of Bottom-Up Heapsort compared to regular Heapsort:

As you can see, for unsorted data, Bottom-Up Heapsort takes up to twice as long as the regular Heapsort, while it takes about the same time for sorted data.

Before we get to the bottom of the cause, let us first examine a smaller section of the diagram:

Bottom-Up Heapsort only becomes slower than the regular Heapsort, starting at about two million elements.

What is the cause of this?

The result of the CountOperations program mentioned above shows that Bottom-Up Heapsort requires fewer compare, read, and write operations than regular heapsort - regardless of the number of elements to be sorted.

Why is it still slower?

Bottom-Up Heapsort is based on the assumption that the root element is always moved down to the leaf level. The branch prediction of the CPU can also make use of this assumption and thus relativize this advantage.

Furthermore, in Bottom-Up Heapsort, we have to go through the tree twice: once from top to bottom and once back to the top. This does not increase the number of operations, but it does affect the access to main memory!

While memory pages only have to be loaded once from the main memory into the CPU cache when traversing the tree once, most memory pages are already removed from the cache and must be reread on the way back if the tree is large enough.

Therefore we approach the speed factor two for sufficiently large trees.

Bottom-Up Heapsort With Expensive Comparison Operations

Bottom-Up Heapsort is optimized to reduce the number of comparisons required. With int primitives, comparisons are not significant, so Bottom-Up Heapsort cannot show its advantages here.

I have, therefore, carried out another test by artificially increasing the cost of the comparison operations. You can find the adapted algorithms in the classes HeapsortSlowComparisons and BottomUpHeapsortSlowComparisons in the GitHub repository.

Bottom-Up Heapsort performs significantly better in this comparison:

Further Characteristics of Heapsort

In the following sections, we look at the space complexity of heapsort, its stability, and parallelizability.

Space Complexity of Heapsort

Heapsort is an in-place sorting method, i.e., no additional memory space is required except for loop and auxiliary variables. The number of these variables is always the same, whether we sort ten elements or ten million. Therefore:

The space complexity of heapsort is: O(1)

It is easy to construct examples that show that elements with the same key can change their position to each other:

When we sort the array [3, 2a, 2b, 1] with Heapsort, we perform the following steps ( 2a and 2b represent two elements with the same key; highlighted in light yellow are the elements that will be swapped in the next step; highlighted in blue are finished elements):

At this point, we can abort because we can already see that the target array will end in [2a, 3], i.e., 2a will end up to the right of 2b in the target array.

In the second step, we swapped the 1 with the 2a according to the algorithm. Could we change the algorithm so that for child nodes with the same key, the parent is not swapped with the left child, but with the right one?

In that case, the array above would be sorted stable because the 1 would not be swapped with the 2a, but with the 2b. And then, the 2b would end up at the second last position of the array.

Let's try this with another input array, with [4, 3, 2a, 2b, 1]:

After step 2, we have reached the state we had before as the initial array, with 2a and 2b having swapped their positions. If we now exchange 1 with the right child in the next step, the same thing happens as above: 2a arrives first in the target array and thus right of 2b.

We have shown counterexamples for both algorithm variants and can therefore state:

Heapsort is not a stable sorting algorithm.

Heapsort Parallelizability

With Heapsort, the whole array is continuously changing, so there are no apparent solutions to parallelize the algorithm.

Comparing Heapsort With Other Efficient Sorting Algorithms

The following diagram shows the UltimateTest results of Heapsort compared to the ones of Quicksort and Merge Sort from the respective articles:

Heapsort is slower than Quicksort by factor 3.6 and slower than Merge Sort by factor 2.4 for randomly distributed input data. For sorted data, heapsort is eight to nine times slower than quicksort and two times slower than Merge Sort.

As shown in the previous section, Quicksort is usually much faster than heapsort.

Due to the O(n²) worst-case time complexity of Quicksort, Heapsort is sometimes preferred to Quicksort in practice.

As shown in the article about Quicksort, if the pivot element is chosen appropriately, the worst case is unlikely to occur. Nevertheless, there is a certain risk that a potential attacker with sufficient knowledge of the Quicksort implementation used can exploit this knowledge to crash or freeze an application with appropriately prepared input data.

Merge Sort is also usually faster than Heapsort. Besides, unlike Heapsort, Merge Sort is stable.

Heapsort has an advantage over Merge Sort in that it does not require additional memory, while Merge Sort requires additional memory in the order of O(n).

Heapsort is an efficient, unstable sorting algorithm with an average, best-case, and worst-case time complexity of O(n log n).

Heapsort is significantly slower than Quicksort and Merge Sort, so Heapsort is less commonly encountered in practice.

You will find more sorting algorithms in the overview of all sorting algorithms and their characteristics in the first part of the article series.

If you liked the article, feel free to share it using one of the share buttons at the end. Would you like to be informed by email when I publish a new article? Then click here to subscribe to the HappyCoders newsletter.

# data[].attributes.attachment_link.title
Heapsort - Algorithm, Source Code, Time Complexity

# data[].attributes.attachment_link.description
How does Heapsort work? With illustrations and source code. How to determine its time complexity (without complicated maths)?

# data[].attributes.body
https://www.fdi.ucm.es/profesor/gmendez/docs/edi0910/_02-Recursion.pdf

# data[].attributes.attachment_link.readable_text
According to the Oxford English Dictionary, "greedy" means having excessive desire for something without considering the effect or damage done.

In computer science, a greedy algorithm is an algorithm that finds a solution to problems in the shortest time possible. It picks the path that seems optimal at the moment without regard for the overall optimization of the solution that would be formed.

Edsger Dijkstra, a computer scientist and mathematician who wanted to calculate a minimum spanning tree, introduced the term "Greedy algorithm". Prim and Kruskal came up with optimization techniques for minimizing cost of graphs.

Many Greedy algorithms were developed to solve graph problems. A graph is a structure made up of edges and vertices.

Greedy vs Not Greedy Algorithms

An algorithm is greedy when the path picked is regarded as the best option based on a specific criterion without considering future consequences. But it typically evaluates feasibility before making a final decision. The correctness of the solution depends on the problem and criteria used.

Example: A graph has various weights and you are to determine the maximum value in the tree. You'd start by searching each node and checking its weight to see if it is the largest value.

There are two approaches to solving this problem: greedy approach or not greedy.

This graph consists of different weights and we need to find the maximum value. We'll apply the two approaches on the graph to get the solution.

Greedy Approach

In the images below, a graph has different numbers in its vertices and the algorithm is meant to select the vertex with the largest number.

Starting from vertex 6, then it's faced with two decisions - which is bigger, 3 or 4? The algorithm picks 4, and then is faced with another decision - which is bigger, 14 or 11. It selects 14, and the algorithm ends.

On the other hand there is a vertex labeled 20 but it is attached to vertex 3 which greedy does not consider as the best choice. It is important to select appropriate criteria for making each immediate decision.

Not Greedy Approach

The "not greedy" approach checks all options before arriving at a final solution, unlike the "greedy approach" which stops once it gets its results.

Starting from vertex 6, then it's faced with two decisions - which is bigger, 3 or 4? The algorithm picks 4, and then is faced with another decision - which is bigger, 14 or 11. It selects 14 and keeps it aside.

Then it runs the process again, starting from vertex 6. It selects the vertex with 3 and checks it. 20 is attached to the vertex 3 and the process stops. Now it compares the two results - 20 and 14. 20 is bigger, so it selects the vertex (3) that carries the largest number and the process ends.

This approach considers many possibilities in finding the better solution.

Characteristics of a Greedy Algorithm

The algorithm solves its problem by finding an optimal solution. This solution can be a maximum or minimum value. It makes choices based on the best option available.

The algorithm is fast and efficient with time complexity of O(n log n) or O(n). Therefore applied in solving large-scale problems.

The search for optimal solution is done without repetition - the algorithm runs once.

It is straightforward and easy to implement.

How to Use Greedy Algorithms

Before applying a greedy algorithm to a problem, you need to ask two questions:

Do you need the best option at the moment from the problem?

Do you need an optimal solution (either minimum or maximum value)?

If your answer to these questions is "Yes", then a greedy algorithm is a good choice to solve your problem.

Procedure

Let's assume you have a problem with a set of numbers and you need to find the minimum value.

You start of by defining the constraint, which in this case is finding the minimum value. Then each number will be scanned and checked on each constraint which serves as a condition to be fulfilled. If the condition is true, the number(s) is selected and returned as the final solution.

Here's a flowchart representation of this process:

Greedy Algorithm Examples

Problem 1 : Activity Selection Problem

This problem contains a set of activities or tasks that need to be completed. Each one has a start and finish time. The algorithm finds the maximum number of activities that can be done in a given time without them overlapping.

Approach to the Problem

We have a list of activities. Each has a start time and finish time.

First, we sort the activities and start time in ascending order using the finish time of each.

Then we start by picking the first activity. We create a new list to store the selected activity.

To choose the next activity, we compare the finish time of the last activity to the start time of the next activity. If the start time of the next activity is greater than the finish time of the last activity, it can be selected. If not we skip this and check the next one.

This process is repeated until all activities are checked. The final solution is a list containing the activities that can be done.

The table below shows a list of activities as well as starting and finish times.

The first step is to sort the finish time in ascending order and arrange the activities with respect to the result.

# data[].attributes.attachment_link.title
What is a Greedy Algorithm? Examples of Greedy Algorithms

# data[].attributes.attachment_link.description
According to the Oxford English Dictionary, "greedy" means having excessive desire for something without considering the effect or damage done. In computer science, a greedy algorithm is an algorithm that finds a solution to problems in the shortest time possible. It picks the path that seems optimal at the

# data[].attributes.headline
Análisis simplificado de algoritmos recursivos

# data[].attributes.subject
Análisis simplificado de algoritmos recursivos

# data[].attributes.attachment_link.readable_text
Recursion is one of the popular problem-solving approaches in data structure and algorithms. Even some problem-solving approaches are totally based on recursion: Decrease and conquer, Divide and conquer, DFS traversal of tree and graph, Backtracking, Top-down approach of dynamic programming, etc. So learning the time complexity analysis of recursion is critical to learn these approaches and improving the performance of recursive algorithms.

Sometimes programmers find it challenging to analyze recursion due to its mathematical details. But working with recursive algorithms becomes easy when we understand various recursive patterns and methods used in the analysis.

What is recurrence relation of a recursive function?

A recurrence relation is an equation that describes a sequence where any term is defined using its previous terms. We use recurrence relations to analyze the time complexity of recursive functions in terms of input size.

In the case of recursion, we solve a problem by breaking it down into smaller subproblems. If the time complexity of a recursive function is T(n), then the time complexity of the smaller subproblems will be defined by the same function, but in terms of the subproblem's input size.

Here is a common approach to writing T(n) if we have k number of subproblems: T(n) = T(Input size of 1st subproblem) + T(Input size of 2nd subproblem) + .... + T(Input size of kth subproblem) + Time complexity of additional operations other than recursive calls.

The above formula provides an approach to defining the recurrence relation of every recursive algorithm. We then solve this recurrence relation and calculate the overall time complexity for recursion in terms of Big-O notation. Let's better understand this via examples of various recurrence relations of popular recursive algorithms.

Decrease by a constant: Reverse an array

We are reversing an n-size array by swapping the first and last values and recursively solving the remaining subproblem of reversing an (n - 2) size array. At each step of recursion, the input size decreases by 2.

The time complexity T(n) = Time complexity of solving an (n - 2) size problem + Time complexity of the swapping operation = T(n - 2) + O(1).

The recurrence relation to reverse an array is T(n) = T(n - 2) + c, where T(1) = c.

Decrease by a constant factor: Binary search

At each step of recursion, we are doing one comparison and decreasing the input size by half. In other words, we are solving the problem of size n using the solution of one subproblem of input size n/2 (based on the comparison, either the left or right sub-problem).

The time complexity T(n) = Time complexity of the n/2 size problem + Time complexity of the comparison operation = T(n/2) + O(1).

The recurrence relation of binary search is T(n) = T(n/2) + c, where T(1) = c.

Dividing into two equal size subproblems: Merge sort

In merge sort, we solve the problem of size n by using the solution of two equal subproblems of input size n/2 and merging the sub-problems' solutions in O(n) time.

The time complexity T(n) = Time complexity of the left sub-problem of size n/2 + Time complexity of the right sub-problem of size n/2 + Time complexity of merging = T(n/2) + T(n/2) + O(n) = 2T(n/2) + cn.

The recurrence relation of merge sort is T(n) = 2T(n/2) + cn, where T(1) = c.

Dividing into two different size subproblems: Quick sort

In quicksort, we divide an array into two subarrays in O(n) time and recursively solve the subproblems. The size of the subarrays depends on the choice of pivot in the partition algorithm. Let k be the number of elements in the left subarray (left of the pivot), and n - k - 1 be the number of elements in the right subarray (right of the pivot). Then, the size of the left subproblem is k, and the size of the right subproblem is n - k - 1.

The time complexity T(n) = Time complexity of the partition algorithm + Time complexity of the left sub-problem of size k + Time complexity of the right sub-problem of size (n - k - 1) = O(n) + T(k) + T(n - k - 1) = T(k) + T(n - k - 1) + cn.

The recurrence relation of quicksort is T(n) = T(k) + T(n - k - 1) + cn, where T(1) = c.

Dividing into more than two subproblems of equal size

Karatsuba algorithm for fast multiplication: Here, we solve integer multiplication problem of size n bits using three sub-problems of size n/2 bits and combine these solutions in O(n) time. Recurrence relation: T(n) = 3*T(n/2) + cn, where T(1) = c.

Strassen's matrix multiplication: Here, we solve matrix multiplication problem of size n using solution of seven sub-problems of size n/2 and combining these solutions in O(n^2) time. Recurrence relation: T(n) = 7*T(n/2) + cn^2, where T(1) = c.

To find the nth Fibonacci number, we recursively solve and add two subproblems of size (n - 1) and (n - 2). Here subproblems are dependent on each other because the value of the nth Fibonacci depends on the (n - 1)th and (n - 2)th Fibonacci, and the value of the (n - 1)th Fibonacci depends on the value of the (n - 2)th Fibonacci, and so on. Such types of dependent subproblems arise in dynamic programming problems.

Time complexity T(n) = Time complexity of finding the (n - 1)th Fibonacci + Time complexity of finding the (n - 2)th Fibonacci + Time complexity of addition operation = T(n - 1) + T(n - 2) + O(1).

Recurrence relation of finding the nth Fibonacci number: T(n) = T(n - 1) + T(n - 2) + c, where T(n) = c for n <= 1.

Steps to analyse time complexity of recursive algorithms

Step 1: Identify input size and smaller subproblems

We first identify the input size of the larger problem.

Then we recognise the total number of smaller sub-problems.

Finally, we identify the input size of smaller sub-problems.

Step 2: Write recurrence relation for the time complexity

We define the time complexity function for the larger problem in terms of the input size. For example, if the input size is n, then the time complexity function would be T(n).

Then we define the time complexity of smaller subproblems. For example, in the case of merge sort, the input size of both subproblems is n/2 each, so the time complexity of each subproblem would be T(n/2).

Next, we analyze the worst-case time complexity of additional operations. For example, in merge sort, the divide (O(1)) and merging process (O(n)) are extra operations.

We add the time complexities of the sub-problems and additional operations to get the equation of the overall time complexity function T(n).

We also define the time complexity of the base case, i.e., the smallest version of the problem. Our solution to the recurrence depends on this, so we need to define it correctly. Think!

Step 3: Solving recurrence relation to get the time complexity

We mainly use the following two methods to solve recurrence relations in algorithms and data structure. We can choose these methods depending on the nature of the recurrence relation. The master method works best to analyse divide and conquer algorithms, but a recursion tree method is always a fundamental approach applied to all types of recursive algorithms.

Method 1: Recursion Tree Method

Method 2: Master Theorem

Method 1: Analysis of recursion using Recursion Tree Method

A recursion tree is a tree diagram of recursive calls where each tree node represents the cost of a certain subproblem. The idea is simple! The time complexity of a recursive function depends on two factors: 1) the total number of recursive calls and 2) the time complexity of additional operations for each recursive call.

So, a recursion tree is a diagram that represents the additional cost for each recursive call in terms of its input size. We should add the extra cost for each recursive call to get the overall time complexity. The best idea is to perform this sum of extra cost level by level.

Steps of recursion time complexity analysis using recursion tree method

Draw the recursion tree for the given recurrence relation.

Calculate the cost of additional operations for each recursive call.

Now calculate the cost of additional operations at each level by adding the cost of each node present at that level.

Finally, add the cost of each level to determine the total cost of recursion. Simplify this expression to get the overall time complexity recursion in terms of big-O notation.

Example: Analysis of merge sort using recursion tree method

Merge sort recurrence relation: T(n) = 2T(n/2) + cn, T(1) = c.

We are dividing the problem of size n into two different sub-problems of size n/2. Here cn represents the extra cost of merging the solution of two smaller sub-problems of size n/2 (merging two smaller sorted arrays of size n/2). One more thing: At each step of recursion, the problem will be divided in half until the size of the sub-problem becomes 1.

1. We start with the term cn as the root, which represents the cost of additional operations at the first level of the recursion tree. The left and right children of the root are the time complexities of the two smaller sub-problems, which are T(n/2). Now we move one level further by expanding T(n/2).

2. At the second level, there are two nodes of size n/2, and the cost of the additional operations at each node is cn/2. So the overall cost of additional operations at the second level of recursion is cn/2 + cn/2 = cn.

3. Similarly, we move to the third level where we have 4 nodes of size n/4, and the cost of the additional operations at each node is cn/4. The total cost at the third level is cn/4 + cn/4 + cn/4 + cn/4 = cn.

4. In a similar way, we continue expanding each internal node in the recursion tree until the problem sizes get down to 1. The bottom level has n nodes, each contributing a cost of c. So the total cost at the last level is cn.

5. In general, level i has 2^i nodes, each contributing a cost of cn/2^i. So the total cost of any ith level is 2^i *(cn/2^i) = cn.

Here, the recursion tree is a full binary tree structure where each node has two children, and each level is completely full. At each level, the input size decreases by a factor of 2, and the recursion tree will terminate at an input size of 1. So the height of the recursion tree h = logn.

The total number of levels in a binary tree = height of the binary tree + 1 = logn + 1.

To compute the total cost represented by the recurrence, we add up the costs of all the levels. The recursion tree has logn + 1 levels, each costing cn. So the total cost = cn * (logn + 1) = cnlogn + cn.

After ignoring the low-order term and the constant c in (cnlogn + cn), the time complexity of merge sort = O(nlogn).

To explore further, you can understand the analysis of binary search and quicksort using the recursion tree method.

Method 2: Analysis of Recursion using Master Theorem

We use the master method for finding time complexity of divide and conquer algorithm that partition an input into smaller subproblems of equal sizes. It is primarily a direct way to get the solution for recurrences that can be transformed to the type: T(n) = aT(n/b) + O(n^k), where a≥1 and b>1.

The master theorem recurrence describes time complexity of an algorithm that divides a problem of size n into a number of subproblems, each of size n/b, where a and b are positive constants. Here a number of subproblems are solved recursively, each in time T(n/b) and O(n^k) is the cost of dividing the problem and combining the solution of subproblems.

There are three cases of recursion analysis using the master theorem:

Case 1: When k < logb(a) then T(n) = O(n^logb(a))

Case 2: When k = logb(a) then T(n) = O(n^k * logn)

Case 3: When k > logb(a) then T(n) = O(n^k)

Special notes

The general recurrence relation of the master theorem is T(n) = aT(n/b) + f(n), where f(n) is an asymptotically positive function. For simplicity of analysis, we assume that f(n) = O(n^k) for some positive constant k. This assumption holds true for most recursive solutions, where the cost of the additional operations is polynomial in n.

The master theorem is derived from the recursion tree method, where the height of the recursion tree is logb(n). However, the mathematical proof of this theorem is beyond the scope of this discussion. If you are interested in learning the proof, you can refer to the CLRS book.

Example 1: Binary search analysis using master theorem

Comparing master theorem recurrence relation with binary search recurrence relation:

Here a = 1, b = 2 (a > 1 and b > 1) k = 0 because n^k = c = Cn^0 => logb(a) = log2(1) = 0 => k = logb(a) We can apply case 2 of the master theorem.

T(n) = O(n^k * log(n)) = O(n^0 * log(n)) = O(logn).

Example 2: Merge sort analysis using master theorem

Comparing master theorem recurrence with merge sort recurrence relation:

Here a = 2, b = 2 (a > 1 and b > 1) O(n^k)= cn = O(n¹) => k = 1 logb(a) = log 2(2) = 1 Hence => k = logb(a) = 1 We can apply case 2 of the master theorem.

Time complexity T(n) = O(n^k * logn) = O(n^1 * logn) = O(nlogn).

Example 3: Divide and conquer idea of finding max and min

Comparing master theorem recurrence with finding max and min recurrence relation:

Here a = 2, b = 2 (a > 1 and b > 1) O(n^k)= cn = O(n^0) => k = 0 logb(a) = log 2(2) = 1 Hence => logb(a) > k We can apply case 1 of the master theorem.

Time complexity T(n) = O(n^logb(a)) = O(n^1) = O(n).

Example 4: Analysis of strassen's matrix multiplication

Comparing master theorem recurrence with strassen's multiplication recurrence:

Here a = 7, b = 2 (a > 1 and b > 1) O(n^k)= cn^2 = O(n^2) => k = 2 logb(a) = log 2(7) = 2.8 (Approximately) Hence => logb(a) > k so we can apply case 1 of the master theorem.

Time complexity T(n) = O(n^logb(a)) = O(n^log2(7)) = O(n^2.81).

Explore analysis of following recursive algorithms

If you have any queries or feedback, please write to us at contact@enjoyalgorithms.com. Enjoy learning, enjoy coding, and enjoy algorithms!

# data[].attributes.attachment_link.title
Time Complexity Analysis of Recursive Function in DSA

# data[].attributes.attachment_link.description
In data structures and algorithms, problem-solving using recursion becomes easy when we understand various methods to analyze recursion time complexity. In this blog, we discuss how to write recurrence relations of various recursive algorithms and steps to analyze the time complexity of recursion using the recursion tree method and the master theorem.

# data[].attributes.attachment_link.readable_text
In this article, you will learn about the "Radix Sort" sorting algorithm. You will learn:

How does Radix Sort work? (Step by step)

How to implement Radix Sort in Java?

What is the time and space complexity of Radix Sort?

What variants of Radix Sort exist?

... and what does the term "radix" mean anyway?

Let's start with the last question:

What is Radix Sort?

"Radix" is the Latin word for "root" - nevertheless, Radix Sort has nothing to do with calculating square roots.

Instead, the "radix" of a number system (also called the "base") refers to the number of digits needed to represent numbers in that number system. The radix in the decimal system is 10, the radix of the binary system is 2, and the radix of the hexadecimal system is 16.

In Radix Sort, we sort the numbers digit by digit - and not, as in most other sorting methods, by comparing two numbers. You can read more about how this works in the following chapter.

Radix Sort Algorithm

The algorithm for Radix Sort is best explained step by step using an example. We want to sort the following numbers:

We will start by looking at the last digit only (there are also Radix Sort variations where you start at the first digit, but we'll get to that later):

We sort the numbers in two phases: a partitioning phase and a collection phase.

Partitioning Phase

For the partitioning, we create ten so-called "buckets", designated with "0" to "9". We distribute the numbers to these buckets according to their last digit. The following image demonstrates how we place the first number, 41, in bucket "1":

The second number, 573, is placed in bucket "3" according to its last digit:

The third number, 3, is also placed in bucket "3":

In the same way, we distribute the remaining numbers to the buckets:

That completes the partitioning phase for the last digit.

Collection Phase

The partitioning phase is followed by the collecting phase. We collect the numbers, bucket by bucket, in ascending order - and within the buckets from left to right (i.e., in the same order as the numbers were entered in the respective bucket) - into a new list.

We start with the bucket with the smallest digit, i.e., bucket 1:

After that, we collect the numbers of the next higher bucket, that's bucket 3:

And finally, the numbers from bucket 6 and then bucket 8:

All buckets are now empty:

In this new list, the numbers are sorted in ascending order by their last digit: 1, 1, 3, 3, 6, 8

Sorting by Tens Place

We repeat the partitioning and collecting phase for the tens place digits. This time, I represent the two phases with only one image each.

In the partitioning phase, we distribute the numbers to the buckets according to their tens place digit:

The tens place digit of one-digit numbers is zero. Accordingly, I have represented the three as "03".

In the collection phase, we again collect the numbers, bucket by bucket:

The numbers are now sorted according to their last two digits: 3, 8, 36, 41, 71, 73, 93

Sorting by Hundreds Place

We repeat the same procedure for the hundreds place. First, the partitioning phase:

And after that, the collection phase:

After the third and final collection phase, the numbers are entirely sorted.

Here again, are the final result without leading zeros:

In the next chapter, we come to the implementation of Radix Sort.

Radix Sort in Java

Radix Sort can be implemented in several ways. We'll start with a simple variant that is very close to the algorithm described. After that, I will show you two alternative implementations.

Variant 1: Radix Sort With Dynamic Lists

We start with an empty sort() method and fill it step by step.

(You can find the final result at the end of this section and in the RadixSortWithDynamicLists class in the GitHub repository of this sorting tutorial series).

Since we need to repeat the two phases (partitioning phase and collecting phase) for each digit, we first need to determine how many digits our numbers have.

We do this by finding the largest number from the array to be sorted and then counting how many times that number can be divided by 10:

Then we sort digit by digit. We write a for loop with the loop variable digitIndex, where 0 stands for the units place, 1 for the tens place, 2 for the hundreds place, and so on.

(In the following listings, I don't print the class anymore, only the methods within the class).

For the next step, we need the buckets to distribute the numbers to. We could use ten ArrayLists for this.

However, it is more elegant to wrap them in a Bucket class. That makes the code more readable and allows us to change the implementation of the buckets later without having to change the rest of the code.

We can create the Bucket class as an inner class inside RadixSortWithDynamicLists:

That was the preparation.

Let's move on to the partitioning phase. We need ten buckets on which to distribute the numbers; we generate them with a createBuckets() method:

After that, we distribute our numbers among the buckets based on the digit at the digitIndex currently under consideration:

The divisor is the number by which we must divide an element so that the rearmost digit is the digit currently under consideration - i.e., 1 for the units place, 10 for the tens place, 100 for the hundreds place, and so on.

We combine the methods of the partitioning phase in a partition() method:

private Bucket[] partition(int[] elements, int digitIndex) { Bucket[] buckets = createBuckets(); distributeToBuckets(elements, digitIndex, buckets); return buckets; }

In the collection phase, all we have to do is join the numbers from the individual buckets:

{ int targetIndex = 0; for (Bucket bucket : buckets) { for (int element : bucket.getElements()) { elements[targetIndex] = element; targetIndex++; } } }

We combine the partition() and collect() methods into a sortByDigit() method:

And now, we close the circle by calling the sortByDigit() method from the digitIndex loop in the sort() method shown at the beginning:

{ int max = getMaximum(elements); int numberOfDigits = getNumberOfDigits(max); for (int digitIndex = 0; digitIndex < numberOfDigits; digitIndex++) { sortByDigit(elements, digitIndex); } }

That completes our Radix Sort implementation.

Here you can see the complete source code again:

By the way, the RadixSortWithDynamicLists class in the GitHub repository is slightly different from the source code printed here:

It implements the SortAlgorithm interface, which allows comparison of different Radix Sort implementations with each other and with the other algorithms of the sorting algorithm series.

The getMaximum() method is placed in the ArrayUtils class.

The getNumberOfDigits() and calculateDivisor() methods are in the RadixSortHelper class and can thus be used in other Radix Sort implementations.

The implementation shown has one shortcoming:

Dynamic lists (i.e., lists that can change size at runtime) are not optimal for performance-critical purposes such as sorting algorithms because adding elements involves some performance overhead (for example, in a linked list, new nodes must be created; in an ArrayList, the array must be recopied into a larger one at certain intervals).

In the next section, I will show you an alternative variant.

Variant 2: Radix Sort with Arrays

We can speed up the implementation significantly (we will compare the performance of the implementations afterward) by using an array instead of an ArrayList for the buckets.

Since arrays have a fixed size, we need to know how many elements a bucket will contain before creating it. We modify our Bucket class as follows and pass the size to its constructor:

To determine how many elements a bucket should contain, we first count the digits at the current digitIndex. The partition() method then looks like this:

We don't need to change the distributeToBuckets() method or any other method shown in variant 1. Good thing we used a Bucket class in the first variant - and not an ArrayList :-)

You can find the complete code of variant 2 in the RadixSortWithArrays class in the GitHub repository.

Let's move on to a third variant.

Variant 3: Radix Sort with Counting Sort

In variant 2, we counted in advance how many elements would be sorted into each bucket. With this information, we can skip the buckets and move the elements directly to their target positions. We do this by applying the general form of Counting Sort.

I won't repeat here how Counting Sort works. I'll show you the implementation right away:

You can also find this code in the GitHub repository, in the RadixSortWithCountingSort class.

Radix Sort Variants

There are two basic variants of Radix Sort, which differ in the order in which we look at the digits of the elements.

LSD Radix Sort

The Radix Sort algorithm shown in the first chapter is called "LSD Radix Sort". LSD stands for "least significant digit". We started sorting at the least significant digit (the ones) and worked our way up, digit by digit, to the most significant digit.

MSD Radix Sort

Alternatively, we can also start at the most significant digit. Accordingly, the second variant is called "MSD Radix Sort".

However, we have to proceed differently than with the LSD variant. Because if we were to sort the entire input list in our initial example first by hundreds place, then by tens place, and finally by units place, the following would happen (I have omitted the buckets in the graphic since we are only concerned with the results after the three collect phases):

Sorting by the tens place and ones place has mixed up the respective previous sortings.

The problem is solved quickly:

After the hundreds place, we must not sort the input list again as a whole, but the hundreds place buckets within themselves. We then sort the resulting tens place buckets by the units place. In other words, we sort the buckets recursively.

MSD Radix Sort - Step by Step

The following diagrams show the recursive MSD Radix Sort procedure step by step using the initial example. Buckets are represented by brackets under the elements. Empty buckets are not shown.

We start with partitioning by hundreds place:

Now, instead of moving from the partitioning phase to the collecting phase, we perform another partitioning phase on each bucket - on the next lower digit, that is, the tens place.

Empty buckets and those containing only one element (such as the 271 and the 836) need not be partitioned further.

Actually, we would now have to partition the buckets by units place. But since none of the tens place buckets contains more than one element, this is unnecessary.

We, therefore, exit the recursion. First, we execute a collection phase on the tens place buckets:

And lastly, we perform the collection phase on the hundreds place buckets:

That completes the sorting.

MSD Radix Sort - Implementation

Like the LSD variant, we can implement MSD Radix Sort with dynamic lists, arrays, and Counting Sort.

I'll show you how to modify the LSD array implementation shown above into an MSD implementation with just a few changes.

Here are once more the sort() and sortByDigit() methods of the RadixSortWithArrays class:

All we have to do now is call the sortByDigit() method for the most significant digit first and insert the recursive call for the next lower digit between the partitioning and collecting phases:

The Bucket.needsToBeSorted() method returns true if the bucket contains at least one element.

You can find the complete code in the RecursiveMsdRadixSortWithArrays class in the GitHub repository.

As an exercise, I'll leave it to you to write an MSD variant for each of the other two LSD implementations (dynamic lists and Counting Sort).

Using Other Bases

So far, we have partitioned according to the decimal system, i.e., with ten buckets. However, we can also work with any other base, for example, with the binary system (2 buckets), the hexadecimal system (16 buckets), or even with a hundred, a thousand, or more buckets.

The higher the base, the more buckets, and the more complex the partitioning phase. On the other hand, the numbers to sort have fewer digits (1,000,000 decimal = F4240 hexadecimal), so altogether fewer partitioning and collecting phases are required. We will determine what this means for performance in the " Radix Sort Runtime" chapter.

How do you implement Radix Sort with a different base?

Basically, we need to replace each occurrence of the number 10 in the source code with the new base. In the RadixSortWithDynamicLists class, "10" occurs in the following methods:

We can replace the "10" in all these places with another base. Better yet, we replace it with a variable so that we can invoke the sorting algorithm with any base.

In the RadixSortWithDynamicListsAndCustomBase class, you can find the corresponding adjustment:

Please note that in the GitHub repository, the getNumberOfDigits() and calculateDivisor() methods are located in the RadixSortHelper class, as other Radix Sort implementations also use them.

In the GitHub repository, you can also find the adapted algorithms for arrays, Counting Sort, and recursive MSD Radix Sort:

Radix Sort Time Complexity

In this chapter, I will show you how to determine the time complexity of Radix Sort. For an introduction to time complexity, see the article " Big O Notation and Time Complexity".

We use the following variables below:

n = the number of elements to sort

k = the maximum key length (number of digit places) of the elements to sort

b = the base (= the number of buckets)

The algorithm iterates over k digit places; for each place, it performs the following operation:

It creates b buckets. The cost of this is constant in each case.

It iterates over all n elements to sort them into the buckets. The cost of calculating a bucket number and inserting an element into a bucket is constant.

It iterates over b buckets and copies a total of n elements from them. The cost for each of these steps is again constant.

We ignore constant parts in the determination of the time complexity. This results in:

The time complexity for Radix Sort is: O(k · (b + n))

The cost is independent of how the input numbers are arranged. Whether they are randomly distributed or pre-sorted makes no difference to the algorithm. Best case, average case, and worst case are, therefore, identical.

The formula looks complicated at first. But two of the three variables are not variable in most situations. For example, if we sort longs with a base of 10, we can replace k with 19 (the maximum possible value for a long is 9,223,372,036,854,775,807) and b with 10.

The formula then becomes O(19 · (10 + n)). We can omit constants; thus, we get:

The time complexity for Radix Sort
with a known maximum length of the elements to sort
and with a fixed base is: O(n)

So, for primitive data types like integer and long (for these, we know the maximum length), Radix Sort has a better time complexity than Quicksort!

You'll find out whether Radix Sort is actually faster in the next chapter. We will measure the runtime of the various Radix Sort implementations and compare them with each other (and with Quicksort).

Radix Sort Runtime

In this chapter, I'll show you the results of some performance tests I ran using the UltimateTest and CompareRadixSorts tools to compare the performance of different algorithms, implementations, and bases.

Runtime of Different Radix Sort Implementations

The first diagram shows the comparison of the different implementations:

As expected, the implementation with dynamic lists performs worst. The remaining three variants are in a neck-and-neck race, which the Counting Sort implementation wins by a narrow margin, closely followed by the array variant.

We can also see the linear running time O(n) in each case, which we predicted in the previous chapter.

Effect of the Base on the Runtime

The second diagram shows how the choice of the base affects the runtime of the array implementation:

We can see that the runtime is significantly better for bases of 100 and 1,000 than for smaller and larger bases.

Let's examine this in a little more detail... The third diagram shows finer gradations of the bases with a fixed number of elements ( n = 5,555,555):

Both too small and too large a base are bad for performance.

A very small base leads to many iterations. A base that is too large leads to fewer iterations but significantly more buckets within the iterations.

A sweet spot shows up at a base of 256.

Radix Sort vs. Quicksort

In the following diagram, you can see the runtimes...

of the Radix Sort array implementation with a base of 256,

of dual-pivot Quicksort combined with insertion sort (the fastest variant we determined in the Quicksort tutorial), and

of the JDK sort method Arrays.sort(), which also implements an optimized dual-pivot Quicksort.

And indeed, Radix Sort is not only faster in theory - O(n) vs. O(n log n) - but also in practice - comparing it with both the home-implemented Quicksort and the even faster JDK Quicksort implementation Arrays.sort().

So if you need to sort int primitives and performance is critical, you should consider using Radix Sort instead of Java's native Arrays.sort(). Feel free to use the implementation from this article.

That is not true for long primitives. For longs, Arrays.sort() is about 50% faster than my Radix Sort implementation.

Other Characteristics of Radix Sort

In this concluding chapter, we consider the space complexity, stability, and parallelizability of Radix Sort, as well as its differences from Counting Sort and Bucket Sort.

Radix Sort Space Complexity

All variants shown in this article require additional memory:

O(b) for the digit counters (not needed in the dynamic lists variant)

O(b) for the bucket references (not required for the counting-sort variant).

O(n) for the contents of the buckets (not needed for the counting-sort variant)

O(n) for an additional target array (only for the Counting Sort variant)

Each variant thus contains at least one O(b) component and at least one O(n) component.

We can therefore conclude:

The space complexity of Radix Sort is: O(b + n)

There is one exception: recursive MSD Radix Sort with base 2 can do without additional memory for the elements by partitioning them in such a way that by exchanging two elements at a time, all elements whose bit is 1 at the currently considered place are pushed to the right side, and all elements whose bit is 0 are pushed to the left side (similar to Quicksort).

Is Radix Sort Stable?

You can read about the definition of stability in sorting methods in the linked introductory article. In short: elements with the same key keep their original order to each other during sorting.

All Radix Sort implementations shown in this article are stable.

In contrast, the in-place MSD Radix Sort variant discussed in the previous section is not stable (analogous to Quicksort).

Parallel Radix Sort

Both Radix Sort variants (LSD and MSD) can be parallelized.

Parallel MSD Radix Sort

With MSD Radix Sort, after the initial partitioning phase, we can sort all the resulting buckets independently in parallel. Thanks to parallel streams, this is very easy to implement in Java:

Here again, is the corresponding sequential code from the RecursiveMsdRadixSortWithArrays class:

for (Bucket bucket : buckets) { if (bucket.needsToBeSorted()) { sortByDigit(bucket.getElements(), digitIndex - 1); } }

And here is the parallelized variant ( ParallelRecursiveMsdRadixSortWithArrays class in the GitHub repository):

Arrays.stream(buckets) .parallel() .forEach( bucket -> { if (bucket.needsToBeSorted()) { sortByDigit(bucket.getElements(), digitIndex - 1); } });

Parallel LSD Radix Sort

To parallelize LSD Radix Sort, we need to put a little more effort:

1. We divide the input array into segments to be processed in parallel (e.g., according to the number of CPU cores).

2. We calculate in parallel per segment how many elements have to be sorted into which buckets.

3. When step 2 is complete for all segments, we compute a) per bucket, the total number of elements, and b) per segment, the initial write positions for each bucket.

4. We distribute the elements from the segments to the buckets in parallel. Using the initial write positions calculated in step 3, we know at which positions within the buckets we must write from which segments.

5. When step 4 is complete for all segments, we compute per bucket the offset in the target array (as prefix sums over the number of elements in the buckets).

6. We collect the elements from the buckets in parallel. Using the offsets calculated in step 5, we know at which position in the target array the elements of a bucket must start.

You can find the source code in the ParallelRadixSortWithArrays class in the GitHub repo. The six steps above are marked in the code with correspondingly numbered comments.

Parallel vs. Sequentiell Radix Sort

The following diagram shows the runtime of the parallel variants compared to the sequential variants on a 6-core i7 CPU:

The parallel variants are only about 2.3 times faster, with 67 million elements. That is not even close to factor 6, partly because parts of the code cannot be executed in parallel and partly because the CPU cores have to exchange a lot of data with the main memory (the input array occupies 1 GB).

If we look at a smaller section of the diagram, things look different:

With "only" half a million elements, the parallel Radix Sort variant with arrays is 5.75 times faster than the sequential variant. The CPU cores are almost entirely utilized. That is because the input array is only 2 MB, and the sorting can take place completely in the CPU's L3 cache.

Radix Sort vs. Counting Sort

Both sorting methods use buckets for sorting. With Counting Sort, we need one bucket for each value. For example, if we wanted to sort integers, we would need about four billion buckets. With Radix Sort, on the other hand, the number of buckets corresponds to the chosen base.

In Radix Sort, we sort iteratively digit by digit; in Counting Sort, we sort the elements in a single iteration.

Counting Sort is therefore primarily suitable for small number spaces.

Radix Sort vs. Bucket Sort

Bucket Sort first distributes items across a given number of buckets such that all items in each bucket are greater than all items in the previous bucket (e.g., 0-99, 100-199, 200-299, etc.).

After that, each bucket is sorted in itself - either recursively with Bucket Sort - or with another sorting method (which exactly is not specified). Finally, the elements from the sorted buckets are joined.

If this sounds familiar to you - you've met one form of Bucket Sort in this article: recursive MSD Radix Sort.

Summary

Radix Sort is a stable sorting algorithm with a general time complexity of O(k · (b + n)), where k is the maximum length of the elements to sort ("key length"), and b is the base.

If the maximum length of the elements to sort is known, and the basis is fixed, then the time complexity is O(n).

For integers, Radix Sort is faster than Quicksort (at least in my test environment). If you need to implement time-critical sorting operations in Java, I recommend you compare Arrays.sort() with an implementation of Radix Sort.

You can find more sorting algorithms in the overview of all sorting algorithms and their characteristics in the first part of the article series.

If you liked the article, please share it using one of the share buttons at the end. Want to be notified by email when I publish a new article? Then click here to join the HappyCoders newsletter.

# data[].attributes.attachment_link.title
Radix Sort - Algorithm, Source Code, Time Complexity

# data[].attributes.attachment_link.description
In this article, you will learn about the "Radix Sort" sorting algorithm. You will learn: How does Radix Sort work? (Step by step) How to implement Radix Sort in Java? What is the time and space complexity of Radix Sort? What variants of Radix Sort exist? ...

# data[].attributes.attachment_link.readable_text
All sorting methods discussed so far in this article series are based on comparing whether two numbers are smaller, larger or equal. Counting Sort is based on a completely different, non-comparison approach.

This article answers the following questions:

How does Counting Sort work?

What is the difference between the simplified form of Counting Sort and its general form?

What does the source code of Counting Sort look like?

How to determine the time complexity of Counting Sort?

Why is Counting Sort almost ten times faster for presorted number sequences than for unsorted ones despite the same number of operations?

Counting Sort Algorithm (Simplified Form)

Instead of comparing elements, Counting Sort counts how often which elements occur in the set to be sorted.

A simplified form of Counting Sort can be used when sorting numbers (e.g., int primitives). To sort objects according to their keys, you will learn about Counting Sort's general form afterward.

The simplified form consists of two phases:

Counting Sort Algorithm - Phase 2: Counting the Elements

First, an auxiliary array is created whose length corresponds to the number range (e.g., an array of size 256 to sort bytes). Then you iterate once over the elements to be sorted, and, for each element, you increment the value in the array at the position corresponding to the element.

Here is an example with the number range 0-9 (i.e., the array to be sorted contains only numbers from 0 to 9).

The following array shall be sorted:

We create an additional array of length 10, initialized with zeros. In the diagram, the array index is displayed below the line:

Now we iterate over the array to be sorted. The first element is a 3 - accordingly, we increase the value in the auxiliary array at position 3 by one:

The second element is a 7. We increment the field at position 7 in the helper array:

Elements 4 and 6 follow - thus, we increase the values at positions 4 and 6 by one each:

The next two elements - the 6 and the 3 - are two elements that have already occurred before. Accordingly, the corresponding fields in the auxiliary array are increased from 1 to 2:

The principle should be clear now. After also increasing the auxiliary array values for the remaining elements, the auxiliary array finally looks like this:

This so-called histogram tells us the following:

The elements to be sorted contain:

We will use this information in phase 2 to rearrange the array to be sorted.

Counting Sort Algorithm - Phase 2: Rearranging the Elements

In phase two, we iterate once over the histogram array. We write the respective array index into the array to be sorted as often as the histogram indicates at the corresponding position.

In the example, we start at position 0 in the auxiliary array. That field contains a 1, so we write the 0 exactly once into the array to be sorted.

(I grayed out the rest of the numbers because they are still in the array, but we don't need them anymore. We now have this information entirely in the histogram.)

At position 1 in the histogram, there is a 0, meaning we skip this field - no 1 is written to the array to be sorted.

Position 2 of the histogram again contains a 1, so we write one 2 into the array to be sorted:

We come to position 3, which contains a 3; so we write three times a 3 into the array:

And so it goes on. We write once the 4, five times the 6, once the 7, twice the 8 and finally once the 9 into the array to be sorted:

The numbers are sorted; the algorithm is completed.

Counting Sort Java Code Example (Simplified Form)

Below you'll find a simple form of the Counting Sort source code - it only works for non-negative int primitives (e.g., for the array from the example above).

First, the findMax() method is used to find the largest element in the array. Then the auxiliary array counts is created of the corresponding size, where the size is one greater than the largest element so we can count the 0 as well.

(For smaller number ranges like byte and short, you can omit to determine the maximum and directly create an array in the size of the corresponding number range.)

In the block commented with "Phase 1", the elements are counted so that the counts array eventually contains the histogram.

In the block commented with "Phase 2", the elements are written back to the array to be sorted in ascending order and according to the histogram's frequency.

You could also determine the maximum using Arrays.stream(elements).max().getAsInt(). But then we would either have to omit the check for negative values or do it in a separate step.

You can find the code in the GitHub repository in the class CountingSortSimple.

Counting Sort Source Code Also for Negative Numbers

If you want to allow negative numbers too, the code gets a bit more complicated because we have to work with a so-called offset to map the number to be sorted to the auxiliary array position.

Calculating the Offset

The offset is: zero minus the smallest number to sort.

If, for example, -5 is the smallest number to be sorted, then the offset is 5, i.e., the index in the auxiliary array is always the number to be sorted plus 5.

For example, the -5 is counted at position -5+5 = 0; the 0 is counted at position 0+5 = 5; the 11 is counted at position 11+5 = 16.

You can find the following source code in the CountingSort class in the GitHub repository. It is similar to the source code shown above, except for the following differences:

The method findMax() is replaced by the method findBoundaries(), which returns not only the maximum but also the minimum value (for small number ranges like byte and short, you can omit to determine the boundaries and directly create an array in the size of the number range).

When accessing the counts array during the counting phase, the -boundaries.min offset is added to the corresponding index (or -Byte.MIN_VALUE or -Short.MIN_VALUE).

When writing back the sorted numbers into the array, the offset is subtracted again by adding boundaries.min (or Byte.MIN_VALUE or Short.MIN_VALUE).

public { private static final int MAX_VALUE_TO_SORT = Integer.MAX_VALUE / 2; private static final int MIN_VALUE_TO_SORT = Integer.MIN_VALUE / 2; { Boundaries boundaries = findBoundaries(elements); int[] counts = new int[boundaries.max - boundaries.min + 1]; for (int element : elements) { counts[element - boundaries.min]++; } int targetPos = 0; for (int i = 0; i < counts.length; i++) { for (int j = 0; j < counts[i]; j++) { elements[targetPos++] = i + boundaries.min; } } } private Boundaries findBoundaries (int[] elements) { int min = Integer.MAX_VALUE; int max = Integer.MIN_VALUE; for (int element : elements) { if (element > MAX_VALUE_TO_SORT) { throw new IllegalArgumentException("Element " + element + " is greater than maximum " + MAX_VALUE_TO_SORT); } if (element < MIN_VALUE_TO_SORT) { throw new IllegalArgumentException("Element " + element + " is less than minimum " + MIN_VALUE_TO_SORT); } if (element > max) { max = element; } if (element < min) { min = element; } } return new Boundaries(min, max); } private static { private final int min; private final int max; { this.min = min; this.max = max; } } }

This variant not only has the advantage of being able to count negative numbers but also occupies less additional memory than the first variant if the number range does not start at 0: For numbers from 1,000 to 2,000, for example, the first variant would need an auxiliary array with 2,001 fields, whereas variant 2 only needs 1,001 fields.

Counting Sort Algorithm (General Form)

You can not only use Counting Sort to sort arrays of primitives (i.e., bytes, ints, longs, doubles, etc.) but also for arrays of objects. For this purpose, we have to extend the algorithm, as described in the following section.

General Algorithm - Phase 1: Counting the Elements

Phase 1, the counting phase, remains more or less unchanged. Instead of the objects themselves, their keys (determined by a getKey() method, for example) are now counted.

The array in the following image references objects whose keys correspond to the numbers in the previous example, i.e., 3, 7, 4, 6, 6, etc.:

Accordingly, the resulting histogram resembles the one from the first example:

General Algorithm - Phase 2: Aggregating the Histogram

Here the difference to the simplified algorithm becomes obvious: We now know that the element with the key 0 occurs once, but we cannot merely write a 0 into the array to be sorted - we instead need the object with the key 0!

To find this efficiently, we first aggregate the values in the histogram. For this purpose, we iterate, starting at index 1, over the auxiliary array and add to each field the left neighboring field's value.

At position 1, we add to the 0 the value of field 0, the 1. The sum is 1:

At position 2, we add to the 1 the 1 from field 1 and get a 2:

To the 3 at position 3, we add the 2 of field 2 - the sum is 5:

And so we continue until we finally add to the 1 in field 9 the 14 from field 8 to get 15:

This aggregated histogram now no longer tells us how often the objects with a specific key occur, but at which position the last element with the corresponding key belongs. The position is 1-based, not 0-based.

For example, the object with key 0 belongs at position 1 (corresponds to index 0 in the array), the object with key 2 at position 2 (array index 1), and the three objects with key 3 at positions 3, 4, and 5 (array indexes 2, 3, 4).

General Algorithm - Phase 3: Writing Back Sorted Objects

To sort the objects, we need an additional array in the size of the input array:

We now iterate backward over the array to be sorted and write each object into the target array to the position indicated by the auxiliary array. We decrement the corresponding value in the auxiliary array by 1 to put the next object with the same key one field further to the left.

Let's start at the far right in the input array - with the object with key 8. In the auxiliary array, position 8 has the value 14. We decrement the value to 13 and copy the object with key 8 to the target array at position 13 (remember: the position information in the auxiliary array is 1-based, so we write at position 13, not 14).

The second object from the right has the key 2. In the auxiliary array, position 2 has the value 2. We decrement the value in the auxiliary array to 1 and copy the object to the target array's corresponding position:

The next object has the key 6. In the auxiliary array, position 6 contains 11. We decrement the value to 10 and copy the object to field 10 in the target array:

Following the same logic, we copy the object with the key 9 to position 14 in the target array:

An additional six follows. In the auxiliary array, position 6 now contains the 10 (after we had decremented the 11). We decrement the value again to 9 and copy the object to position 9 in the target array, i.e., to the left of the other object with key 6:

We repeat these steps for all elements and finally reach the object with the key 3. Field 3 in the auxiliary array now contains a 3. We decrement this to 2 and copy the object to position 2, the target array's last free position:

The objects are sorted; the algorithm is finished.

Counting Sort Java Code Example (General Form)

The following code demonstrates the general form of Counting Sort for simplicity's sake using int primitives. The findMax() method is equal to the one in the first source code example, so I omitted it here.

You can find the source code in the CountingSortGeneral class in the GitHub repository..

Counting Sort Time Complexity

The time complexity of Counting Sort is easy to determine due to the very simple algorithm.

Let n be the number of elements to sort and k the size of the number range.

The algorithm contains one or more loops that iterate to n and one loop that iterates to k.

Constant factors are irrelevant for the time complexity; therefore:

The time complexity of Counting Sort is: O(n + k)

Runtime of the Java Counting Sort Example

The GitHub repository contains the UltimateTest program, which allows us to measure the speed of Counting Sort (and all the other sorting algorithms in this article series).

The following table shows the time needed to sort unsorted and ascending and descending presorted elements for the given number of elements n, which in these measurements also corresponds to the size of the number range k:

You can find the complete result in the file Test_Results_Counting_Sort.log. The following diagram shows the measurements graphically:

You can see the following:

Pre-sorted output sequences with half a billion elements are sorted about nine times faster than unsorted ones.

For presorted input sequences, the measurements correspond to the expected linear time complexity O(n + k).

For unsorted input sequences, the measurements are slightly higher: When the array size doubles, the time required increases by a factor of about 2.1 to 2.2.

Input sequences sorted in descending order are sorted minimally slower than those pre-sorted in ascending order.

If elements are not actually sorted but counted and entirely rearranged, shouldn't the initial order do not affect the time needed for sorting!?

Using the program CountOperations, we can measure how many operations are needed for sorting. And indeed, the result confirms the assumption (see file CountOperations_Counting_Sort.log):

The number of operations is independent of the initial order of the elements.

The number of operations corresponds to the expected time complexity O(n + k), thus increasing linearly with the number of elements to sort and the size of the number range.

Then what causes these deviating measurements? You will find explanations in the following sections.

Why Is Counting Sort Faster for Presorted Elements Than for Unsorted Ones?

An auxiliary array with half a billion elements is 2 GB in size. If its elements are incremented in random order, a new cache line (typically 64 bytes) must be exchanged between RAM and CPU cache for almost every element. The larger the array, the lower the probability that the required cache line is in the CPU cache.

In contrast, if the array is incremented from front to back (or from back to front), 16 consecutive int values can be loaded from and written to the RAM in a single 64-byte block.

This does not quite achieve an acceleration by factor 16, but at least one by factor nine.

Why Doesn't Counting Sort Achieve Linear Time Complexity for Unsorted Output Sequences in Practice?

The larger the array to be sorted, the higher the ratio of cache misses to cache hits when accessing the auxiliary array (because the size of the CPU cache remains the same).

So with an array twice as big, we don't have twice as many cache misses, but a little more than twice as many. Accordingly, the time required increases by a little more than a factor of two.

Why Is Counting Sort Faster for Items Sorted in Ascending Order Than for Items Sorted in Descending Order?

If elements are sorted in ascending order, they are not changed and do not have to be written back to RAM. With elements sorted in descending order, every element of the array changes, so the whole array has to be written back into RAM once.

Further Characteristics of Counting Sort

In this chapter, we determine the space complexity, stability, and parallelizability of Counting Sort.

Space Complexity of Counting Sort

The simplified algorithm requires an additional array of size k; therefore:

The space complexity of the simplified counting sort algorithm is: O(k)

In addition to the auxiliary array of size k, the general algorithm requires a temporary target array of size n; thus:

The space complexity of the general counting sort algorithm is: O(n + k)

Stability of Counting Sort

In Phase 3, the general form of the Counting Sort algorithm iterates from right to left over the input array, copying objects with the same key also from right to left into the output array. Thus:

Counting Sort is a stable sorting algorithm.

Parallelizability of Counting Sort

Counting Sort can be parallelized by dividing the input array into as many partitions as there are processors available.

In phase 1, each processor counts the elements of "its" partition in a separate auxiliary array.

In phase 2, all auxiliary arrays are added up to one.

In phase 3, each processor copies the elements of "its" partition to the target array. The decrementing and reading of the fields in the auxiliary array must be done atomically.

Due to parallelization, it can no longer be guaranteed that elements with the same key are copied to the target array in their original order.

Parallel Counting Sort is therefore not stable.

Counting Sort is a very efficient, stable sorting algorithm with a time and space complexity of O(n + k).

Counting Sort is mainly used for small number ranges. In the JDK, for example, for:

byte arrays with more than 64 elements (for fewer elements, Insertion Sort is used)

short or char arrays with more than 1,750 Elementen (for fewer elements, Insertion Sort or Dual-Pivot Quicksort is used)

If you liked the article, feel free to share it using one of the share buttons at the end. Would you like to be informed by e-mail when I publish a new article? Then click here to subscribe to the HappyCoders newsletter.

# data[].attributes.attachment_link.title
Counting Sort - Algorithm, Source Code, Time Complexity

# data[].attributes.attachment_link.description
All sorting methods discussed so far in this article series are based on comparing whether two numbers are smaller, larger or equal. Counting Sort is based on a completely different, non-comparison approach. This article answers the following questions: How does Counting Sort work?

# data[].attributes.body
https://youtu.be/o9pT9cWzbnI

# data[].attributes.body
Escalabilidad algorítmica, Árboles B+

# data[].attributes.attachment_link.readable_text
Los nodos hojas del índice están almacenados en un orden aleatorio, es decir su posición en el disco no corresponde a la posición lógica según el orden del índice. Es lo mismo que una guía telefónica con las hojas mezcladas. Si buscas "Sánchez" pero primero abres la guía telefónica por "Rodríguez", nada nos asegura que Rodríguez siga a Sánchez. Una base de datos necesita una segunda estructura para encontrar rápidamente los datos dentro de las hojas mezcladas: un árbol de búsqueda equilibrado, o sea, un B-Tree.

Figura 1.2 La estructura del B-tree

Figura 1.2 muestra un ejemplo de un índice con 30 entradas. La lista doblemente enlazada establece el orden lógico entre los nodos hojas. La raíz y sus ramas permiten hacer una búsqueda rápida entre los nodos hojas.

En la ilustración destaca el nodo rama y sus nodos hojas a los que hace referencia. Cada entrada del nodo rama corresponde al valor más grande respecto a su nodo hoja. Por ejemplo, el valor más grande en el primer nodo hoja es 46, que está almacenado en su propia entrada del nodo rama. Lo mismo sucede para todos los nodos hojas, que al final de su nodo rama tienen los valores 46, 53, 57 y 83. De acuerdo con este plan, un nivel de rama está construido hasta que todos los nodos hojas estén cubiertos por un nodo rama.

El siguiente nivel está construido de manera similar, pero encima del primer nivel de rama. Se repite este procedimiento hasta que todas las llaves llenan un único nodo, el nodo raíz. La estructura es un árbol de búsqueda equilibrado porque la profundidad del árbol es idéntica en cada posición; la distancia entre el nodo raíz y los nodos hojas es idéntica en todas las partes del árbol.

Una vez creado el índice, la base de datos lo mantiene automáticamente. Se aplican cada insert, delete y update al índice y se conserva el árbol equilibrado, lo que genera una sobrecarga de mantenimiento para las operaciones de escritura. El Capítulo 8, "Modificando los datos", explica este tema con más detalles.

Figura 1.3 Recorrido de un índice B-Tree

Figura 1.3 es una muestra de un fragmento de un índice para ilustrar la búsqueda de la llave "57". El recorrido del árbol empieza desde el nodo raíz del lado izquierdo. Cada entrada esta procesada en el orden ascendente hasta encontrar un valor superior o igual (>=) al valor buscado (57). En esta ilustración, se trata del valor 83. La base de datos sigue la referencia sobre el nodo rama correspondiente y repite el proceso hasta que el recorrido del árbol alcanza un nodo hoja.

El recorrido del árbol es una operación muy eficiente. De hecho es tan eficiente que me refiero a ella como la primera potencia de la indexación. Funciona de manera casi instantánea incluso con grandes volúmenes de datos. Ello se debe principalmente a la característica equilibrada del árbol, lo que permite tener acceso a todos los elementos con el mismo número de etapas, y en segundo lugar, al crecimiento logarítmico de la profundidad del árbol. Eso significa que la profundidad del árbol crece lentamente en comparación al número de hojas. Hay índices reales con millones de registros que tienen una profundidad de cuatro o cinco. Es poco común encontrar una profundidad de seis. El apartado sobre la "Escalabilidad logarítmica" lo describe más en detalle.
Si te gusta mi manera de explicar, te encantará mi libro.

# data[].attributes.attachment_link.title
El árbol de búsqueda equilibrado (B-Tree) en las bases de datos SQL

# data[].attributes.attachment_link.description
Los nodos hojas del índice están almacenados en un orden aleatorio, es decir su posición en el disco no corresponde a la posición lógica según el orden del índice. Es lo mismo que una guía telefónica con las hojas mezcladas. Si buscas "Sánchez" pero primero abres la guía telefónica por "Rodríguez", nada nos asegura que Rodríguez siga a Sánchez.

# data[].attributes.body
B tree and B+ tree indexing sqlite

# data[].attributes.attachment_link.title
How Database B-Tree Indexing Works - DZone Database

# data[].attributes.attachment_link.description
Take a look into how database indexing works on a database.

# data[].attributes.attachment_link.readable_text
The binary heap data structure is heap implementation. These are often shown as an array object that can be viewed as nearly complete binary tree built out of a given set of data. The heap data structure is also used in the construction of a priority queue. The complete binary tree maps the binary tree structure into array indices, as shown in the figure below. Each array index represents a node.

The node's parent, left, and right child can be expressed as

There are two kinds of binary heaps: max-heap and min-heap. Both types of heaps satisfy a certain heap property.

Max-heap Property:
If is an array representation of a heap, then in max-heap which means that a node can't have a greater value than its parent. In a max-heap, the largest element is stored at the root, and the minimum elements are in the leaves.

Min-heap Property:
Similarly, if is an array representation of a heap, then in min-heap which means that a parent node can't have a greater value than its children. Thus, the minimum element is located at the root, and the maximum elements are located in the leaves.

Both min-heap and max-heap can be used to implement a heapsort, but this wiki will discuss heapsort in terms of max-heaps. To convert to min-heap, just change the problem around to use min-heaps and ensure that the min-heap property holds.

# data[].attributes.attachment_link.title
Heap Sort | Brilliant Math & Science Wiki

# data[].attributes.attachment_link.description
The binary heap data structure is heap implementation. These are often shown as an array object that can be viewed as nearly complete binary tree built out of a given set of data. The heap data structure is also used in the construction of a priority queue.

# data[].attributes.attachment_link.description
Heapsort is similar to selection sort-we're repeatedly choosing the largest item and moving it to the end of our array. But we use a heap to get the largest item more quickly. O(n*lg(n)) time in total.

# data[].attributes.attachment_link.readable_text
Dijkstra's algorithm finds a shortest path tree from a single source node, by building a set of nodes that have minimum distance from the source.

The graph has the following:

This is done by initializing three values:

, an array of distances from the source node to each node in the graph, initialized the following way: () = 0; and for all other nodes , () = . This is done at the beginning because as the algorithm proceeds, the from the source to each node in the graph will be recalculated and finalized when the shortest distance to is found

, a queue of all nodes in the graph. At the end of the algorithm's progress, will be empty.

, an empty set, to indicate which nodes the algorithm has visited. At the end of the algorithm's run, will contain all the nodes of the graph.

The algorithm proceeds as follows:

1. While is not empty, pop the node , that is not already in , from with the smallest (). In the first run, source node will be chosen because () was initialized to 0. In the next run, the next node with the smallest value is chosen.

2. Add node to , to indicate that has been visited

3. Update values of adjacent nodes of the current node as follows: for each new adjacent node ,

# data[].attributes.attachment_link.title
Dijkstra's Shortest Path Algorithm | Brilliant Math & Science Wiki

# data[].attributes.attachment_link.description
One algorithm for finding the shortest path from a starting node to a target node in a weighted graph is Dijkstra's algorithm. The algorithm creates a tree of shortest paths from the starting vertex, the source, to all other points in the graph.

# data[].attributes.attachment_link.readable_text
Consider the highway system of the eastern coast of the United States. A road inspector is given the task of writing reports about the current condition of each highway. What would be the most economical way for him to traverse all the cities? The problem can be modeled as a graph.

In fact, since graphs are dots and lines , they look like road maps. The dots are called vertices or nodes and the lines are called edges. They may have a value assigned to them (weighted) or they may just be a mere indicator of the existence of a path (unweighted). More formally, a graph can be defined as follows:

Because graphs are so pervasive, it is useful to define different types of graphs. The following are the most common types of graphs:

Undirected graph: An undirected graph is one in which its edges have no orientations, i.e. no direction is associated with any edge. The edges and are equivalent.

Directed graph: A directed graph or digraph consists of a set of vertices (or nodes) and a set of edges (or arcs) such that each edge is associated with an ordered pair of vertices. If there is an edge , it is completely distinct from the edge .

List all the edges and vertices of the undirected graph in the figure above.

The graph consists of the set of vertices = {Massachusetts, Maine, Connecticut, New York, Maryland, New Jersey}.

Its edges are {(Maine,Massachusetts) , (Massachusetts, Connecticut) , (Connecticut,New York), (New York,Maine), (New York,Massachusetts), (New Jersey, Maine),(Maryland, New York), (Maine, Maryland)}.

Note that since the graph is undirected, the order of the tuples in denoting the edges is unimportant.

# data[].attributes.attachment_link.description
There are many systems in the natural world and in society that are amenable to mathematical and computational modeling. However, not everything is easily codified as a system of particles with coordinates and momenta. Some systems and problems such as social networks, ecologies, and genetic regulatory schemes are intrinsically divorced from spacetime descriptions, and instead are more naturally expressed as graphs that reflect their topological properties.

# data[].attributes.body
https://runestone.academy/ns/books/published/pythoned/Graphs/OrdenamientoTopologico.html

# data[].attributes.attachment_link.readable_text
Para demostrar que los científicos de la computación pueden convertir casi cualquier cosa en un problema de grafos, consideremos el difícil problema de batir un montón de panqueques. La receta es realmente muy simple: 1 huevo, 1 taza de mezcla de panqueques, 1 cucharada de aceite y \(3 \over 4\) de una taza de leche. Para hacer panqueques usted debe calentar la plancha, mezclar todos los ingredientes y derramar la mezcla sobre una plancha caliente. Cuando los panqueques empiecen a burbujear, déles vuelta y deje que se cocinen hasta que estén dorados en la parte de abajo. Antes de comer sus panqueques, usted querrá calentar un poco de jarabe dulce. La ilustra este proceso como un grafo.

Lo difícil al preparar panqueques es saber qué hacer primero. Como se puede ver en la , usted puede empezar calentando la plancha o añadiendo cualquiera de los ingredientes a la mezcla de panqueques. Para ayudarnos a decidir el orden preciso en el que debemos hacer cada uno de los pasos requeridos para hacer nuestros panqueques recurriremos a un algoritmo de grafos llamado el ordenamiento topológico.

Un ordenamiento topológico toma un grafo acíclico dirigido y produce un ordenamiento lineal de todos sus vértices de tal manera que si el grafo \(G\) contiene una arista \((v,w)\) entonces el vértice \(v\) está, en el orden, antes del vértice \(w\). Los grafos acíclicos dirigidos se usan en muchas aplicaciones para indicar la precedencia de los eventos. Hacer panqueques es sólo un ejemplo; otros ejemplos incluyen planificaciones de proyectos de software, grafos de precedencia para optimizar las consultas de bases de datos y la multiplicación de matrices.

El ordenamiento topológico es una adaptación simple pero útil de una búsqueda en profundidad. El algoritmo para el ordenamiento topológico es el siguiente:

1. Llamar a para algún grafo . La principal razón por la que queremos invocar a la búsqueda en profundidad es para calcular los tiempos de finalización para cada uno de los vértices.

2. Almacenar los vértices en una lista en orden decreciente según el tiempo de finalización.

3. Devolver la lista ordenada como resultado del ordenamiento topológico.

La muestra el bosque de profundidad construido por para el grafo de preparación de panqueques mostrado en la .

Por último, la muestra los resultados de aplicar el algoritmo de ordenamiento topológico a nuestro grafo. Ahora se ha eliminado toda ambigüedad y sabemos exactamente el orden de los pasos para hacer panqueques.

You have attempted of activities on this page

# data[].attributes.attachment_link.title
7.17. Ordenamiento topológico - Solución de problemas con algoritmos y estructuras de datos

# data[].attributes.attachment_link.description
Para demostrar que los científicos de la computación pueden convertir casi cualquier cosa en un problema de grafos, consideremos el difícil problema de batir un montón de panqueques. La receta es realmente muy simple: 1 huevo, 1 taza de mezcla de panqueques, 1 cucharada de aceite y \(3 \over 4\) de una taza de leche.

# data[].attributes.body
https://www.oracle.com/mx/database/what-is-database/

# data[].attributes.body
http://compsci.hunter.cuny.edu/~sweiss/course_materials/csci493.65/lecture_notes/

# data[].attributes.body
https://bisqwit.iki.fi/story/howto/openmp/

# data[].attributes.body
https://www.youtube.com/watch?v=0fbnyS_lHW4

# data[].attributes.body
https://www.youtube.com/watch?v=cdLNKUoMc6c

# data[].attributes.body
https://www.youtube.com/watch?v=o9pT9cWzbnI

# data[].attributes.body
https://phoenixnap.com/blog/what-is-yaml-with-examples

# data[].attributes.body
https://youtu.be/_gsXXfWotFY

# data[].attributes.body
https://youtu.be/E_V71Ejz3f4

# data[].attributes.attachment_link.readable_text
In computer science, counting sort is an algorithm for sorting a collection of objects according to keys that are small integers; that is, it is an integer sorting algorithm. It operates by counting the number of objects that have each distinct key value, and using arithmetic on those counts to determine the positions of each key value in the output sequence. Its running time is linear in the number of items and the difference between the maximum and minimum key values, so it is only suitable for direct use in situations where the variation in keys is not significantly greater than the number of items. However, it is often used as a subroutine in another sorting algorithm, radix sort, that can handle larger keys more efficiently.

Because counting sort uses key values as indexes into an array, it is not a comparison sort, and the Ω( n log n) lower bound for comparison sorting does not apply to it. Bucket sort may be used for many of the same tasks as counting sort, with a similar time analysis; however, compared to counting sort, bucket sort requires linked lists, dynamic arrays or a large amount of preallocated memory to hold the sets of items within each bucket, whereas counting sort instead stores a single number (the count of items) per bucket.

In the most general case, the input to counting sort consists of a collection of n items, each of which has a non-negative integer key whose maximum value is at most k. In some descriptions of counting sort, the input to be sorted is assumed to be more simply a sequence of integers itself, but this simplification does not accommodate many applications of counting sort. For instance, when used as a subroutine in radix sort, the keys for each call to counting sort are individual digits of larger item keys; it would not suffice to return only a sorted list of the key digits, separated from the items.

In applications such as in radix sort, a bound on the maximum key value k will be known in advance, and can be assumed to be part of the input to the algorithm. However, if the value of k is not already known then it may be computed, as a first step, by an additional loop over the data to determine the maximum key value that actually occurs within the data.

The output is an array of the items, in order by their keys. Because of the application to radix sorting, it is important for counting sort to be a stable sort: if two items have the same key as each other, they should have the same relative position in the output as they did in the input.

In pseudocode, the algorithm may be expressed as:

Here input is the input array to be sorted, key returns the numeric key of each item in the input array, count is an auxiliary array used first to store the numbers of items with each key, and then (after the second loop) to store the positions where items with each key should be placed, k is the maximum value of the non-negative key values and output is the sorted output array.

In summary, the algorithm loops over the items in the first loop, computing a histogram of the number of times each key occurs within the input collection. After that, it then performs a prefix sum computation on count to determine, for each key, the position range where the items having that key should be placed in; i.e. items of key should be placed starting in position count[ ]. This is done through the second loop. Finally, it loops over the items again in the third loop, moving each item into its sorted position in the output array.[1][2][3] The relative order of items with equal keys is preserved here; i.e., this is a stable sort.

Because the algorithm uses only simple for loops, without recursion or subroutine calls, it is straightforward to analyze. The initialization of the count array, and the second for loop which performs a prefix sum on the count array, each iterate at most k + 1 times and therefore take time. The other two for loops, and the initialization of the output array, each take time. Therefore, the time for the whole algorithm is the sum of the times for these steps, .

Because it uses arrays of length k + 1 and n, the total space usage of the algorithm is also . For problem instances in which the maximum key value is significantly smaller than the number of items, counting sort can be highly space-efficient, as the only storage it uses other than its input and output arrays is the Count array which uses space .

If each item to be sorted is itself an integer, and used as key as well, then the second and third loops of counting sort can be combined; in the second loop, instead of computing the position where items with key i should be placed in the output, simply append Count[i] copies of the number i to the output.

This algorithm may also be used to eliminate duplicate keys, by replacing the Count array with a bit vector that stores a one for a key that is present in the input and a zero for a key that is not present. If additionally the items are the integer keys themselves, both second and third loops can be omitted entirely and the bit vector will itself serve as output, representing the values as offsets of the non- zero entries, added to the range's lowest value. Thus the keys are sorted and the duplicates are eliminated in this variant just by being placed into the bit array.

For data in which the maximum key size is significantly smaller than the number of data items, counting sort may be parallelized by splitting the input into subarrays of approximately equal size, processing each subarray in parallel to generate a separate count array for each subarray, and then merging the count arrays. When used as part of a parallel radix sort algorithm, the key size (base of the radix representation) should be chosen to match the size of the split subarrays. The simplicity of the counting sort algorithm and its use of the easily parallelizable prefix sum primitive also make it usable in more fine-grained parallel algorithms.

As described, counting sort is not an in-place algorithm; even disregarding the count array, it needs separate input and output arrays. It is possible to modify the algorithm so that it places the items into sorted order within the same array that was given to it as the input, using only the count array as auxiliary storage; however, the modified in-place version of counting sort is not stable.

Although radix sorting itself dates back far longer, counting sort, and its application to radix sorting, were both invented by Harold H. Seward in 1954.

# data[].attributes.attachment_link.description
In computer science, counting sort is an algorithm for sorting a collection of objects according to keys that are small integers; that is, it is an integer sorting algorithm. It operates by counting the number of objects that have each distinct key value, and using arithmetic on those counts to determine the positions of each key value in the output sequence.

# data[].attributes.attachment_link.readable_text
Counting sheep! Uh, wait, that's not right - I mean counting SORT!

If there's one question that every developer finds themselves asking on a daily basis, it must be this: is it possible to make this better? In almost every context, we seem to ask ourselves some variation of this question. Usually, we're concerned with finding the solution, then getting it out of our heads and into a text editor, whiteboard, or down on a piece of paper. Eventually, we start transforming that idea into code, and the code is pretty terrible the first time around.

And that's okay! Kent Beck's mantra of "Make it work, make it right, make it fast" holds itself to be true for this very reason. But at some point, once we've made it work and made it right, we find ourselves asking: can I make it fast? Can I make it better?

I love sorting things, but there's one glaring problem that I've noticed over the past few weeks - and perhaps you've noticed it too. It's really hard to sort efficiently! We've learned a lot of different ways to sort things: six different ways, to be exact. Most of them had a quadratic runtime, and a few of them had a linearithmic runtime. But, we've never gotten any better than that, have we?

This got me thinking: is it possible to do even better? As it turns out, most of the time, the answer to this question in the context of sorting algorithms is no. It's nearly impossible to do better than linearithmic runtime, or O(n log n) in Big O Notation. The keyword here is, of course, nearly. Because, every once in awhile, we can do better. And today, we're finally going to learn how!

As we near the end of our deep-dive into sorting algorithms, we're going to change gears a bit. We've looked a lot of comparison-based sorting algorithms so far, but the last few algorithms we'll look at are going to be quite different. For starters, they'll be faster than the ones we've been learning about, but they are also limited in how we can use them.

When I say "limited", what I mean is that we're limited in what types of elements we can sort using these algorithms. But this will make more sense when we start looking at an example. Let's start with a definition first!

The first algorithm to wrap our heads around is something called the counting sort algorithm. Don't be fooled by the name - it's not nearly as boring and mathematical as it sounds!

The counting sort algorithm is unique in that it can only be implemented on integers. This is part of what limits this algorithm's usability - and it's possibly part of the reason that you may have never heard or encountered it before! Counting sort takes in a range of integers to be sorted. It uses the range of the integers (for example, the range of integers between 0-100), and counts the number of times each unique number appears in the unsorted input.

The algorithm leverages the fact that it knows beforehand the range of elements to be sorted, and the fact that all the elements are integers, and builds up a "count" array, which it uses to tally up how many numbers appear in the unsorted collection. It then uses some math and a partial hashing function to map the elements to keys in the duplicated "count" array.

Phew, okay, that was a handful of words to get through. And maybe it didn't make all that much sense? Don't worry if it didn't; I tend to think that counting sort makes very little sense as just a definition. Out of all the algorithms we've learned about thus far, I think this one can be the most confusing - unless you see it in action.

Before we sink our heels into an example, let's quickly cover the key things we should always keep in mind about counting sort.

First and foremost, we should only ever think of using or implementing counting sort if the items we want to sort are all integers.

Second, we need to be sure that we know the range of our input integers. As we'll see in the next few sections, counting sort leans heavily on knowing the range of the input; if we don't know how small and how big our input integers are, we can't really use counting sort.

Third, even if we know the range of our input integers, we still need to think a bit about what our integers are; that is to say, what the range between the smallest input integer and the largest input integer could possibly be. If we had 5 elements to sort, but the range of the input values were between 0 and 10,000, counting sort wouldn't work that well, since it has to create a "count" array.

These rules are great to know, but they become so much more obvious with an example. So, let's take a look at one now!

We know that we can only implement counting sort on an integer input, and we know that the range between these elements should stay pretty small. In our example, which is illustrated below, we're sorting an array with just eight elements: [9, 4, 1, 7, 9, 1, 2, 0]. Notice that there are some duplicate values, which is totally okay!

In this situation, we know that the data we're sorting will always range between 0 and 9.

We'll start by creating the "count array", which is highlighted in pink. Remember, we'll need this in order to do the work of tallying up how many of each number (between 0-9) we have in our input data. After we've done the work of tallying, we'll do the work of sorting!

Once we create our "count array", we will be able to use the indexes of the array to map to the elements themselves. When we initialize our "count array", every single index starts off with an initial value of 0. Then, we'll need to do the work of populating our "count array" with the tallied up elements from our unsorted, input array.

So, how are we going to do this, exactly? Well, what we really care about at the moment is just figuring out how many times a single number appears in our input array. In other words, we want to know how many 0's appear in the array, how many 1's, how many 2's, how many 3's, and so on, until we get to the number 9, which is the last, maximum value in our input array. Doing this work is just about as simple as it sounds: we can iterate through the array and keep count of how many times we've seen a certain number, until we've iterated through the entire input dataset.

If you've read about hash tables and hashing functions, you might already be able to guess how this is going to go. Let's see if you're right! In order to make this a bit more obvious, I've redrawn the arrays to be vertical, so that we can see the hashing work more clearly:

Cool! Notice how we're iterating through our unsorted array, and keeping a tallying of how many times we see each number.

For example, the first element that we start with is the number 9. We find index 9 in the count array, which is currently set to 0, and we increment it by one. Then we move on to look at the next element: 4. We find index 4 in the count array, and increment it from 0 to 1. We do the same thing for the next two elements: 1 and 7.

Eventually, we hit another instance of 9. So, we find index 9 in the count array; remember that we already saw an instance of 9, so the value of index 9 in the count array is currently set to 1. But no worries! We have duplicates in this array, and that's fine - we'll just increment it from 1 to 2. Problem solved. We can continue on our merry way, until we've iterated through the entire input array.

Effectively, as we iterate through the unsorted integer array, we're counting each element and keeping track of that count at the corresponding index of the "count array" by incrementing the value at the index that maps to the appropriate number.

Great! We now have a count array that is populated with tallies of how many times each number, ranging from 0-9, appears in our unsorted integer array.

The next step involves some math - but don't worry, it won't be too difficult! All we need to is accumulatively add each pair of consecutive values in our count array.

For example, we'll add the element at index 0 to the element at index 1 in our count array. In this case, these two elements are 1 and 2. Since 1 + 2 = 3, we'll update the index of 1 element to be the sum of the previous element.

We'll continue to build up our count array by adding the previous index's value to the next, for the remainder of the array. As we continue to add accumulatively, our count array is transformed as a result.

Our count array now looks like: [1, 3, 4, 4, 5, 5, 5, 6, 6, 8].

Next, we'll need to shift over our array by one index. In order to do the work of shifting, we can just iterate through our array, and increment the index of each element.

After shifting over our count array by one index, it would look like this:

After shifting, our count array now looks like: [0, 1, 3, 4, 4, 5, 5, 5, 6, 6]. If we think about this a bit more deeply, it starts to (hopefully) make more sense why this step is important. Remember that the value at each index tells us where the first instance of that integer will appear in our sorted array. For example, the value at index 9 in our count array is 6. In other words, this tells us that the first instance of 9 will occur at index 6 of our sorted array. Another way to think about this is that there were be six elements that will appear before the number 9 in our sorted array.

As I discovered through my own trial and error, if we skip this step, our array might still end up in sorted order, but we'll have one extra element added to it, and it'll be just slightly off. So we should be particularly careful to not forget the step of shifting!

Okay, we're down to the last step now: translating our count array into our new, sorted array. The first thing that we'll do is create a new array that will hold our sorted elements. Once we have that, we can actually start the work of sorting. Let's look at all three arrays - our original array, our count array, and our new sorted array, as we do this next step.

First, we'll want to iterate through our original array, and find the index that corresponds to the value we're looking at. Then, we'll look at the tallied count at that index.

For example, we start looking at our first element in the original array: 9. We'll go to our count array, find index 9, and the value at that that index. In this case, it's the number 6.

We'll find the index 6 in our new array, and put a 9 there. And finally, we'll increment the value at the corresponding index in the count array; you can see that I've crossed out the number 6 and replaced it with the number 7 in our count array. We do this increment step after we've sorted the first instance of the number 9.

We can see how this same step is repeated with the next couple elements in the array, and how each time, as we sort, we increment the number at the corresponding index in the count array. The increment step is crucial in order to be able to sort duplicate values, and in order to maintain the order of those elements as they appear in the unsorted array.

Eventually, if we continue doing this as we iterate through the entire array, we'll end up with a new, sorted array, that looks like this:

Nice! Our array = sorted. And our situation = handled. Good work, team.

Okay, we did a pretty detailed walk through of the counting sort algorithm the previous section! Before we look at a quick code example, let's review the basic steps of counting sort.

1. To start, we must create a "count array", which we'll populate by tallying up (or hashing) all the elements in the original array by how many times they appear in the unsorted array.

2. Next, we'll accumulatively add up the values in the populated count array, building it up as we go along.

3. Then, we'll shift over all the elements in the array by incrementing the index of each value by one.

4. Finally, we'll create a new sorted array, which will be the same length as our original array. We'll iterate over our original array, and translate the values over to our new array by using our count array, incrementing our count array value as we continue to sort. In this step, we're effectively using a version of a hashing function, and using our count array as a way to translate values from the unsorted array into the new, sorted one.

Okay, let's see this algorithm in action! Below is an abridged version of Rosetta Code's JavaScript implementation of counting sort. Remember how I mentioned earlier that we need to know the range of the integers in our input array? We can see why in the code below; the maximum and minimum values often end up being parameters in the counting sort function itself, so if we don't know these values, we (quite literally) can't run the algorithm!

Okay, let's run this code and see what happens. I've added some console.log's to make it a bit more obvious as to what's actually going on here. We'll try sorting the exact same array we used in our example: [9, 4, 1, 7, 9, 1, 2, 0]. Notice that we're passing in 0 as our minimum value and 9 as our maximum value, which is the range of the integers that we're trying to sort.

Interesting! We can see our count array being logged out when it is first initialized, and we can see it after we've done the work of tallying. Notice that we do need to create a new array, count, to keep track of our tallying. However, unlike bubble sort or even quick sort, as we append sorted items to our original array, we're doing the same amount of work each time, and we only need to iterate once over each element. You can see the single-iteration over the array happening in the loop on lines 24-32.

And this is what exemplifies exactly what makes counting sort so efficient.

Even though we need some extra space for the count array, and we have to do some slightly complex operations on that count array, the overall amount of time that this algorithm takes is, for the most part, linear. In other words, the time complexity of this algorithm depends directly on how many elements we have to sort.

This also explains a little bit of why the range and input size of our unsorted array matters so much! The space complexity of counting sort hinges directly upon the range of integers that we have to sort. Remember that we create a count array as a result of this algorithm, and the larger our range (for example, 0-9, or 1-100, or even 0-10,000!), the larger our count array will end up being. This can be pretty terrible if we want to sort just 5 or 10 elements, but we have to create a massive array in order to do that! In such cases, counting sort will be Very Bad News™.

In truth, the space-time complexity of counting sort really amounts to a combination of both the number of elements to be sorted, n, and the range between the largest and smallest element, or k. The true Big O notation of counting sort is O(n + k). However, counting sort is generally only ever used if k isn't larger than n; in other words, if the range of input values isn't greater than the number of values to be sorted. In that scenario, the complexity of counting sort is much closer to O(n), making it a linear sorting algorithm.

So how else does counting sort compare to other sorting algorithms we've explored so far?

Well, we already know that, when implemented correctly, counting sort runs in close to linear time. Because the algorithm requires a duplicate array (and doesn't always operate on the original array), it's classified as an out-of-place algorithm. However, it does manage to maintain the order of the elements as they appeared in the original array, making it a sorting algorithm. This ends up being pretty important and useful, and is another advantage of the counting sort algorithm. It also doesn't require any external memory and can sort small amounts of data in memory. Finally, counting sort is unlike other algorithms we've looked at in that it is a non-comparison algorithm. And since we've learned that it is iterative in its implementation, we know that it is non-recursive.

You might not find yourself using counting sort all that often in practice, but it's good to know that linear sorting is possible: as long as you know when to use it. Happy counting!

Even though we might not use counting sort all that frequently, the basics of this particular algorithm come up a lot, especially since it is one of the few algorithms that can run in linear time. It's helpful to understand how it works in the long run, and if you want to learn more about it, there are some good resources that dive into the mathematics behind the steps we learned about today. Get counting with the links below!

1. Counting Sort, Growing With the Web

2. Counting Sort, GeeksForGeeks

3. Linear Sorting, David S. Babcock

4. Sorting in Linear Time (slides), Professor Roger Crawfis

5. Counting Sort Visualization, Professor David Galles

6. Counting Sort, Department of Computer Science, MIT

7. Counting Sort In Minutes, CS Dojo

# data[].attributes.attachment_link.description
If there's one question that every developer finds themselves asking on a daily basis, it must be this: is it possible to make this better? In almost every context, we seem to ask ourselves some variation of this question.

# data[].attributes.attachment_link.readable_text
Here is what it means for an array to be sorted.

By convention, empty arrays and singleton arrays (arrays consisting of only one element) are always sorted. This is a key point for the base case of many sorting algorithms.

Counting sort assumes that each of the input elements in a list has a key value ranging from to , for some integer . For each element in the list, counting sort determines the number of elements that are less than it. Counting sort can use this information to place the element directly into the correct slot of the output array.

Counting sort starts by going through , and for each element , it goes to the index of that has the same value as (so it goes to ) and increments the value of by one. This means that if has seven 's in its list, after counting sort has gone through all elements of , the value at will be . Similarly, if has two 's, after counting sort has gone through all of the elements of , (using 0 indexing) will be equal to . In this step, keeps track of how many elements in there are that have the same value of a particular index in . In other words, the indices of correspond to the values of elements in , and the values in correspond to the total number of times that a value in appears in .

This animation illustrates counting sort:

Here is one way to implement counting sort in Python:

Counting sort is efficient if the range of input data, , is not significantly greater than the number of objects to be sorted, .

# data[].attributes.attachment_link.title
Counting Sort | Brilliant Math & Science Wiki

# data[].attributes.attachment_link.description
Counting sort is an efficient algorithm for sorting an array of elements that each have a nonnegative integer key, for example, an array, sometimes called a list, of positive integers could have keys that are just the value of the integer as the key, or a list of words could have keys assigned to them by some scheme mapping the alphabet to integers (to sort in alphabetical order, for instance).

# data[].attributes.attachment_link.keywords[]
quick sort with median-of-three partitioning : sort « collections « java tutorial

# data[].attributes.attachment_link.title
Quick sort with median-of-three partitioning : Sort " Collections " Java Tutorial

# data[].attributes.attachment_link.description
Quick sort with median-of-three partitioning : Sort " Collections " Java Tutorial

# data[].attributes.attachment_link.provider_name
Educative: Interactive Courses for Software Developers

# data[].attributes.body
Visualización

# data[].attributes.attachment_link.readable_text
In computer science, heapsort is a comparison-based sorting algorithm. Heapsort can be thought of as an improved selection sort: like selection sort, heapsort divides its input into a sorted and an unsorted region, and it iteratively shrinks the unsorted region by extracting the largest element from it and inserting it into the sorted region. Unlike selection sort, heapsort does not waste time with a linear-time scan of the unsorted region; rather, heap sort maintains the unsorted region in a heap data structure to more quickly find the largest element in each step.

Although somewhat slower in practice on most machines than a well-implemented quicksort, it has the advantage of a more favorable worst-case runtime. Heapsort is an in-place algorithm, but it is not a stable sort.

Heapsort was invented by J. W. J. Williams in 1964. This was also the birth of the heap, presented already by Williams as a useful data structure in its own right. In the same year, R. W. Floyd published an improved version that could sort an array in-place, continuing his earlier research into the treesort algorithm.

The heapsort algorithm can be divided into two parts.

In the first step, a heap is built out of the data (see Binary heap § Building a heap). The heap is often placed in an array with the layout of a complete binary tree. The complete binary tree maps the binary tree structure into the array indices; each array index represents a node; the index of the node's parent, left child branch, or right child branch are simple expressions. For a zero-based array, the root node is stored at index 0; if i is the index of the current node, then

  iParent(i)     = floor((i-1) / 2) where floor functions map a real number to the smallest leading integer.
  iLeftChild(i)  = 2*i + 1
  iRightChild(i) = 2*i + 2

In the second step, a sorted array is created by repeatedly removing the largest element from the heap (the root of the heap), and inserting it into the array. The heap is updated after each removal to maintain the heap property. Once all objects have been removed from the heap, the result is a sorted array.

Heapsort can be performed in place. The array can be split into two parts, the sorted array and the heap. The storage of heaps as arrays is diagrammed here. The heap's invariant is preserved after each extraction, so the only cost is that of extraction.

The Heapsort algorithm involves preparing the list by first turning it into a max heap. The algorithm then repeatedly swaps the first value of the list with the last value, decreasing the range of values considered in the heap operation by one, and sifting the new first value into its position in the heap. This repeats until the range of considered values is one value in length.

The steps are:

1. Call the buildMaxHeap() function on the list. Also referred to as heapify(), this builds a heap from a list in O(n) operations.

2. Swap the first element of the list with the final element. Decrease the considered range of the list by one.

3. Call the siftDown() function on the list to sift the new first element to its appropriate index in the heap.

4. Go to step (2) unless the considered range of the list is one element.

The buildMaxHeap() operation is run once, and is O(O(log n) in performance. The siftDown() function is n), and is called n times. Therefore, the performance of this algorithm is .

Pseudocode

The following is a simple way to implement the algorithm in pseudocode. Arrays are zero-based and swap is used to exchange two elements of the array. Movement 'down' means from the root towards the leaves, or from lower indices to higher. Note that during the sort, the largest element is at the root of the heap at a[0], while at the end of the sort, the largest element is in a[end].

procedure heapsort(a, count) isinput: an unordered array a of length count(Build the heap in array a so that largest value is at the root) heapify(a, count)(The following loop maintains the invariants that a[0:end] is a heap and every elementbeyond end is greater than everything before it (so a[end:count] is in sorted order)) end ← count - 1while end > 0 do(a[0] is the root and largest value. The swap moves it in front of the sorted elements.) swap(a[end], a[0])(the heap size is reduced by one) end ← end - 1(the swap ruined the heap property, so restore it) siftDown(a, 0, end)

The sorting routine uses two subroutines, heapify and siftDown. The former is the common in-place heap construction routine, while the latter is a common subroutine for implementing heapify.

(Put elements of 'a' in heap order, in-place)procedure heapify(a, count) is(start is assigned the index in 'a' of the last parent node)(the last element in a 0-based array is at index count-1; find the parent of that element) start ← iParent(count-1)while start ≥ 0 do(sift down the node at index 'start' to the proper place such that all nodes below the start index are in heap order) siftDown(a, start, count - 1)(go to the next parent node) start ← start - 1(after sifting down the root all nodes/elements are in heap order)(Repair the heap whose root element is at index 'start', assuming the heaps rooted at its children are valid)procedure siftDown(a, start, end) is root ← startwhile iLeftChild(root) ≤ end do (While the root has at least one child) child ← iLeftChild(root)(Left child of root) swap ← root(Keeps track of child to swap with)if a[swap] < a[child] then swap ← child(If there is a right child and that child is greater)if child+1 ≤ end and a[swap] < a[child+1] then swap ← child + 1if swap = root then(The root holds the largest element. Since we assume the heaps rooted at the children are valid, this means that we are done.)returnelse swap(a[root], a[swap]) root ← swap(repeat to continue sifting down the child now)

The heapify procedure can be thought of as building a heap from the bottom up by successively sifting downward to establish the heap property. An alternative version (shown below) that builds the heap top-down and sifts upward may be simpler to understand. This siftUp version can be visualized as starting with an empty heap and successively inserting elements, whereas the siftDown version given above treats the entire input array as a full but "broken" heap and "repairs" it starting from the last non-trivial sub-heap (that is, the last parent node).

Also, the siftDown version of heapify , while the siftUp version given below has time complexity due to its equivalence with inserting each element, one at a time, into an empty heap. This may seem counter-intuitive since, at a glance, it is apparent that the former only makes half as many calls to its logarithmic-time sifting function as the latter; i.e., they seem to differ only by a constant factor, which never affects asymptotic analysis.

To grasp the intuition behind this difference in complexity, note that the number of swaps that may occur during any one siftUp call increases with the depth of the node on which the call is made. The crux is that there are many (exponentially many) more "deep" nodes than there are "shallow" nodes in a heap, so that siftUp may have its full logarithmic running-time on the approximately linear number of calls made on the nodes at or near the "bottom" of the heap. On the other hand, the number of swaps that may occur during any one siftDown call decreases as the depth of the node on which the call is made increases. Thus, when the siftDown heapify begins and is calling siftDown on the bottom and most numerous node-layers, each sifting call will incur, at most, a number of swaps equal to the "height" (from the bottom of the heap) of the node on which the sifting call is made. In other words, about half the calls to siftDown will have at most only one swap, then about a quarter of the calls will have at most two swaps, etc.

The heapsort algorithm itself has time complexity using either version of heapify.

procedure heapify(a,count) is(end is assigned the index of the first (left) child of the root) end := 1while end < count(sift up the node at index end to the proper place such that all nodes above the end index are in heap order) siftUp(a, 0, end) end := end + 1(after sifting up the last node all nodes are in heap order)procedure siftUp(a, start, end) isinput:  start represents the limit of how far up the heap to sift.end is the node to sift up. child := endwhile child > start parent := iParent(child)if a[parent] < a[child] then (out of max-heap order) swap(a[parent], a[child]) child := parent(repeat to continue sifting up the parent now)elsereturn

Floyd's heap construction

The most important variation to the basic algorithm, which is included in all practical implementations, is a heap-construction algorithm by Floyd which runs in time and uses siftdown rather than siftup, avoiding the need to implement siftup at all.

Rather than starting with a trivial heap and repeatedly adding leaves, Floyd's algorithm starts with the leaves, observing that they are trivial but valid heaps by themselves, and then adds parents. Starting with element n/2 and working backwards, each internal node is made the root of a valid heap by sifting down. The last step is sifting down the first element, after which the entire array obeys the heap property.

The worst-case number of comparisons during the Floyd's heap-construction phase of Heapsort is known to be equal to , where is the number of 1 bits in the binary representation of n and is number of trailing 0 bits.

The standard implementation of Floyd's heap-construction algorithm causes a large number of cache misses once the size of the data exceeds that of the CPU cache. Much better performance on large data sets can be obtained by merging in depth-first order, combining subheaps as soon as possible, rather than combining all subheaps on one level before proceeding to the one above.

Bottom-up heapsort

Bottom-up heapsort is a variant which reduces the number of comparisons required by a significant factor. While ordinary heapsort requires comparisons worst-case and on average, the bottom-up variant requires comparisons on average, and in the worst case.

If comparisons are cheap (e.g. integer keys) then the difference is unimportant, as top-down heapsort compares values that have already been loaded from memory. If, however, comparisons require a function call or other complex logic, then bottom-up heapsort is advantageous.

This is accomplished by improving the siftDown procedure. The change improves the linear-time heap-building phase somewhat, but is more significant in the second phase. Like ordinary heapsort, each iteration of the second phase extracts the top of the heap, a[0], and fills the gap it leaves with , then sifts this latter element down the heap. But this element comes from the lowest level of the heap, meaning it is one of the smallest elements in the heap, so the sift-down will likely take many steps to move it back down. In ordinary heapsort, each step of the sift-down requires two comparisons, to find the minimum of three elements: the new node and its two children.

Bottom-up heapsort instead finds the path of largest children to the leaf level of the tree (as if it were inserting −∞) using only one comparison per level. Put another way, it finds a leaf which has the property that it and all of its ancestors are greater than or equal to their siblings. (In the absence of equal keys, this leaf is unique.) Then, from this leaf, it searches upward (using one comparison per level) for the correct position in that path to insert . This is the same location as ordinary heapsort finds, and requires the same number of exchanges to perform the insert, but fewer comparisons are required to find that location.

Because it goes all the way to the bottom and then comes back up, it is called heapsort with bounce by some authors.

function leafSearch(a, i, end) is j ← iwhile iRightChild(j) ≤ end do(Determine which of j's two children is the greater)if a[iRightChild(j)] > a[iLeftChild(j)] then j ← iRightChild(j)else j ← iLeftChild(j)(At the last level, there might be only one child)if iLeftChild(j) ≤ end then j ← iLeftChild(j)return j

The return value of the leafSearch is used in the modified siftDown routine:

procedure siftDown(a, i, end) is j ← leafSearch(a, i, end)while a[i] > a[j] do j ← iParent(j) x ← a[j] a[j] ← a[i]while j > i do swap x, a[iParent(j)] j ← iParent(j)

Bottom-up heapsort was announced as beating quicksort (with median-of-three pivot selection) on arrays of size ≥16000.

A 2008 re-evaluation of this algorithm showed it to be no faster than ordinary heapsort for integer keys, presumably because modern branch prediction nullifies the cost of the predictable comparisons which bottom-up heapsort manages to avoid.

A further refinement does a binary search in the path to the selected leaf, and sorts in a worst case of comparisons, approaching the information-theoretic lower bound of comparisons.

A variant which uses two extra bits per internal node ( n−1 bits total for an n-element heap) to cache information about which child is greater (two bits are required to store three cases: left, right, and unknown) uses less than compares.

Other variations

Ternary heapsort uses a ternary heap instead of a binary heap; that is, each element in the heap has three children. It is more complicated to program, but does a constant number of times fewer swap and comparison operations. This is because each sift-down step in a ternary heap requires three comparisons and one swap, whereas in a binary heap two comparisons and one swap are required. Two levels in a ternary heap cover 3² = 9 elements, doing more work with the same number of comparisons as three levels in the binary heap, which only cover 2³ = 8.[citation needed ] This is primarily of academic interest, as the additional complexity is not worth the minor savings, and bottom-up heapsort beats both.

The smoothsort algorithm is a variation of heapsort developed by Edsger Dijkstra in 1981. Like heapsort, smoothsort's upper bound is . The advantage of smoothsort is that it comes closer to time if the input is already sorted to some degree, whereas heapsort averages regardless of the initial sorted state. Due to its complexity, smoothsort is rarely used.[citation needed ]

Levcopoulos and Petersson describe a variation of heapsort based on a heap of Cartesian trees. First, a Cartesian tree is built from the input in time, and its root is placed in a 1-element binary heap. Then we repeatedly extract the minimum from the binary heap, output the tree's root element, and add its left and right children (if any) which are themselves Cartesian trees, to the binary heap. As they show, if the input is already nearly sorted, the Cartesian trees will be very unbalanced, with few nodes having left and right children, resulting in the binary heap remaining small, and allowing the algorithm to sort more quickly than for inputs that are already nearly sorted.

Several variants such as weak heapsort require comparisons in the worst case, close to the theoretical minimum, using one extra bit of state per node. While this extra bit makes the algorithms not truly in-place, if space for it can be found inside the element, these algorithms are simple and efficient, but still slower than binary heaps if key comparisons are cheap enough (e.g. integer keys) that a constant factor does not matter.

Katajainen's "ultimate heapsort" requires no extra storage, performs comparisons, and a similar number of element moves. It is, however, even more complex and not justified unless comparisons are very expensive.

Heapsort primarily competes with quicksort, another very efficient general purpose nearly-in-place comparison-based sort algorithm.

Quicksort is typically somewhat faster due to some factors[which? ], but the worst-case running time for quicksort is , which is unacceptable for large data sets and can be deliberately triggered given enough knowledge of the implementation, creating a security risk. See quicksort for a detailed discussion of this problem and possible solutions.

Thus, because of the upper bound on heapsort's running time and constant upper bound on its auxiliary storage, embedded systems with real-time constraints or systems concerned with security often use heapsort, such as the Linux kernel.

Heapsort also competes with Ω( merge sort, which has the same time bounds. Merge sort requires n) auxiliary space, but heapsort requires only a constant amount. Heapsort typically runs faster in practice on machines with small or slow data caches, and does not require as much external memory. On the other hand, merge sort has several advantages over heapsort:

Merge sort on arrays has considerably better data cache performance, often outperforming heapsort on modern desktop computers because merge sort frequently accesses contiguous memory locations (good locality of reference); heapsort references are spread throughout the heap.

Heapsort is not a stable sort; merge sort is stable.

Merge sort parallelizes well and can achieve close to linear speedup with a trivial implementation; heapsort is not an obvious candidate for a parallel algorithm.

Merge sort can be adapted to operate on singly linked lists with O(1) extra space. Heapsort can be adapted to operate on doubly linked lists with only O(1) extra space overhead.[citation needed ]

Merge sort is used in external sorting; heapsort is not. Locality of reference is the issue.

Introsort is an alternative to heapsort that combines quicksort and heapsort to retain advantages of both: worst case speed of heapsort and average speed of quicksort.

Let { 6, 5, 3, 1, 8, 7, 2, 4 } be the list that we want to sort from the smallest to the largest. (NOTE, for 'Building the Heap' step: Larger nodes don't stay below smaller node parents. They are swapped with parents, and then recursively checked if another swap is needed, to keep larger numbers above smaller numbers on the heap binary tree.)

1. ^ Skiena, Steven (2008). "Searching and Sorting". The Algorithm Design Manual. Springer. p. 109. doi:10.1007/978-1-84800-070-4_4. ISBN 978-1-84800-069-8. [H]eapsort is nothing but an implementation of selection sort using the right data structure.

2. Brass, Peter (2008). Advanced Data Structures. Cambridge University Press. p. 209. ISBN 978-0-521-88037-4.

3. ^ Suchenek, Marek A. (2012), "Elementary Yet Precise Worst-Case Analysis of Floyd's Heap-Construction Program", Fundamenta Informaticae, 120 (1): 75-92, doi:10.3233/FI-2012-751

4. Bojesen, Jesper; Katajainen, Jyrki; Spork, Maz (2000). "Performance Engineering Case Study: Heap Construction" (PostScript). ACM Journal of Experimental Algorithmics. 5 (15): 15-es. CiteSeerX 10.1.1.35.3248. doi:10.1145/351827.384257. Alternate PDF source.

5. ^ Chen, Jingsen; Edelkamp, Stefan; Elmasry, Amr; Katajainen, Jyrki (27-31 August 2012). In-place Heap Construction with Optimized Comparisons, Moves, and Cache Misses (PDF). 37th international conference on Mathematical Foundations of Computer Science. Bratislava, Slovakia. pp. 259-270. doi:10.1007/978-3-642-32589-2_25. ISBN 978-3-642-32588-5. See particularly Fig. 3.

6. ^ Wegener, Ingo (13 September 1993). "BOTTOM-UP HEAPSORT , a new variant of beating, on an average, (if n is not very small)" (PDF). Theoretical Computer Science. 118 (1): 81-98. doi:10.1016/0304-3975(93)90364-y. Although this is a reprint of work first published in 1990 (at the Mathematical Foundations of Computer Science conference), the technique was published by Carlsson in 1987.

7. ^ Fleischer, Rudolf (February 1994). "A tight lower bound for the worst case of Bottom-Up-Heapsort" (PDF). Algorithmica. 11 (2): 104-115. doi:10.1007/bf01182770. hdl:11858/00-001M-0000-0014-7B02-C. Also available as Fleischer, Rudolf (April 1991). A tight lower bound for the worst case of Bottom-Up-Heapsort (PDF) (Technical report). MPI-INF. MPI-I-91-104.

8. Mehlhorn, Kurt; Sanders, Peter (2008). "Priority Queues" (PDF). Algorithms and Data Structures: The Basic Toolbox. Springer. p. 142. ISBN 978-3-540-77977-3.

9. McDiarmid, C.J.H.; Reed, B.A. (September 1989). "Building heaps fast" (PDF). Journal of Algorithms. 10 (3): 352-365. doi:10.1016/0196-6774(89)90033-3.

10. ^ Moret, Bernard; Shapiro, Henry D. (1991). "8.6 Heapsort". Algorithms from P to NP Volume 1: Design and Efficiency. Benjamin/Cummings. p. 528. ISBN 0-8053-8008-6. For lack of a better name we call this enhanced program 'heapsort with bounce.'

11. Carlsson, Scante (March 1987). "A variant of heapsort with almost optimal number of comparisons" (PDF). Information Processing Letters. 24 (4): 247-250. doi:10.1016/0020-0190(87)90142-6.

12. "Data Structures Using Pascal", 1991, page 405,^ [full citation needed ][author missing ][ISBN missing ] gives a ternary heapsort as a student exercise. "Write a sorting routine similar to the heapsort except that it uses a ternary heap."

13. ^ Dijkstra, Edsger W. Smoothsort - an alternative to sorting in situ (EWD-796a) (PDF). E.W. Dijkstra Archive. Center for American History, University of Texas at Austin. (transcription)

14. ^ Schwartz, Keith (27 December 2010). "CartesianTreeSort.hh". Archive of Interesting Code.

15. ^ Katajainen, Jyrki (23 September 2013). Seeking for the best priority queue: Lessons learnt. Algorithm Engineering (Seminar 13391). Dagstuhl. pp. 19-20, 24.

16. ^ Katajainen, Jyrki (2-3 February 1998). The Ultimate Heapsort. Computing: the 4th Australasian Theory Symposium. Australian Computer Science Communications. 20 (3). Perth. pp. 87-96.

17. ^ https://github.com/torvalds/linux/blob/master/lib/sort.c Linux kernel source

Williams, J. W. J. (1964), "Algorithm 232 - Heapsort", Communications of the ACM, 7 (6): 347-348, doi:10.1145/512274.512284

Floyd, Robert W. (1964), "Algorithm 245 - Treesort 3", Communications of the ACM, 7 (12): 701, doi:10.1145/355588.365103

Carlsson, Svante (1987), "Average-case results on heapsort", BIT, 27 (1): 2-17, doi:10.1007/bf01937350

Knuth, Donald (1997), "§5.2.3, Sorting by Selection", Sorting and Searching, The Art of Computer Programming, 3 (third ed.), Addison-Wesley, pp. 144-155, ISBN 978-0-201-89685-5

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, Second Edition. MIT Press and McGraw-Hill, 2001. ISBN 0-262-03293-7. Chapters 6 and 7 Respectively: Heapsort and Priority Queues

A PDF of Dijkstra's original paper on Smoothsort

Heaps and Heapsort Tutorial by David Carlson, St. Vincent College

# data[].attributes.attachment_link.description
In computer science, heapsort is a comparison-based sorting algorithm. Heapsort can be thought of as an improved selection sort: like selection sort, heapsort divides its input into a sorted and an unsorted region, and it iteratively shrinks the unsorted region by extracting the largest element from it and inserting it into the sorted region.

# data[].attributes.attachment_link.readable_text
A binary heap is a heap data structure that takes the form of a binary tree. Binary heaps are a common way of implementing priority queues. The binary heap was introduced by J. W. J. Williams in 1964, as a data structure for heapsort.

A binary heap is defined as a binary tree with two additional constraints:

Shape property: a binary heap is a complete binary tree; that is, all levels of the tree, except possibly the last one (deepest) are fully filled, and, if the last level of the tree is not complete, the nodes of that level are filled from left to right.

Heap property: the key stored in each node is either greater than or equal to (≥) or less than or equal to (≤) the keys in the node's children, according to some total order.

Heaps where the parent key is greater than or equal to (≥) the child keys are called max-heaps; those where it is less than or equal to (≤) are called min-heaps. Efficient ( logarithmic time) algorithms are known for the two operations needed to implement a priority queue on a binary heap: inserting an element, and removing the smallest or largest element from a min-heap or max-heap, respectively. Binary heaps are also commonly employed in the heapsort sorting algorithm, which is an in-place algorithm because binary heaps can be implemented as an implicit data structure, storing keys in an array and using their relative positions within that array to represent child-parent relationships.

Both the insert and remove operations modify the heap to conform to the shape property first, by adding or removing from the end of the heap. Then the heap property is restored by traversing up or down the heap. Both operations take O(log n) time.

To add an element to a heap, we can perform this algorithm:

1. Add the element to the bottom level of the heap at the leftmost open space.

2. Compare the added element with its parent; if they are in the correct order, stop.

3. If not, swap the element with its parent and return to the previous step.

Steps 2 and 3, which restore the heap property by comparing and possibly swapping a node with its parent, are called the up-heap operation (also known as bubble-up, percolate-up, sift-up, trickle-up, swim-up, heapify-up, or cascade-up).

The number of operations required depends only on the number of levels the new element must rise to satisfy the heap property. Thus, the insertion operation has a worst-case time complexity of O(log n). For a random heap, and for repeated insertions, the insertion operation has an average-case complexity of O(1).

As an example of binary heap insertion, say we have a max-heap

and we want to add the number 15 to the heap. We first place the 15 in the position marked by the X. However, the heap property is violated since 15 > 8, so we need to swap the 15 and the 8. So, we have the heap looking as follows after the first swap:

However the heap property is still violated since 15 > 11, so we need to swap again:

which is a valid max-heap. There is no need to check the left child after this final step: at the start, the max-heap was valid, meaning the root was already greater than its left child, so replacing the root with an even greater value will maintain the property that each node is greater than its children (11 > 5; if 15 > 11, and 11 > 5, then 15 > 5, because of the transitive relation).

The procedure for deleting the root from the heap (effectively extracting the maximum element in a max-heap or the minimum element in a min-heap) while retaining the heap property is as follows:

1. Replace the root of the heap with the last element on the last level.

2. Compare the new root with its children; if they are in the correct order, stop.

3. If not, swap the element with one of its children and return to the previous step. (Swap with its smaller child in a min-heap and its larger child in a max-heap.)

Steps 2 and 3, which restore the heap property by comparing and possibly swapping a node with one of its children, are called the down-heap (also known as bubble-down, percolate-down, sift-down, sink-down, trickle down, heapify-down, cascade-down, extract-min or extract-max, or simply heapify) operation.

So, if we have the same max-heap as before

We remove the 11 and replace it with the 4.

Now the heap property is violated since 8 is greater than 4. In this case, swapping the two elements, 4 and 8, is enough to restore the heap property and we need not swap elements further:

The downward-moving node is swapped with the larger of its children in a max-heap (in a min-heap it would be swapped with its smaller child), until it satisfies the heap property in its new position. This functionality is achieved by the Max-Heapify function as defined below in pseudocode for an array-backed heap A of length length( A). Note that A is indexed starting at 1.

For the above algorithm to correctly re-heapify the array, no nodes besides the node at index i and its two direct children can violate the heap property. The down-heap operation (without the preceding swap) can also be used to modify the value of the root, even when an element is not being deleted.

In the worst case, the new root has to be swapped with its child on each level until it reaches the bottom level of the heap, meaning that the delete operation has a time complexity relative to the height of the tree, or O(log n).

Inserting an element then extracting from the heap can be done more efficiently than simply calling the insert and extract functions defined above, which would involve both an upheap and downheap operation. Instead, we can do just a downheap operation, as follows:

1. Compare whether the item we're pushing or the peeked top of the heap is greater (assuming a max heap)

2. If the root of the heap is greater:

1. Replace the root with the new item

2. Down-heapify starting from the root

3. Else, return the item we're pushing

Python provides such a function for insertion then extraction called "heappushpop", which is paraphrased below. The heap array is assumed at have its first element at index 1.

// Push a new item to a (max) heap and then extract the root of the resulting heap.
//heap: an array representing the heap, indexed at 1 //item: an element to insert // Returns the greater of the two betweenitem and the root of heap.Push-Pop(heap: List<T>, item: T) -> T:if heap is not empty and heap[1] > item then: // < if min heapswap heap[1] and item _downheap(heap starting from index 1)return item

A similar function can be defined for popping and then inserting, which in Python is called "heapreplace":

// Extract the root of the heap, and push a new item
//heap: an array representing the heap, indexed at 1 //item: an element to insert // Returns the current root ofheapReplace(heap: List<T>, item: T) -> T:swap heap[1] and item _downheap(heap starting from index 1)return item

Finding an arbitrary element takes O(n) time. This can be reduced to amortized O(1) time if we have a hash table mapping heap elements to heap indices or element pointers.

Deleting an arbitrary element can be done as follows:

1. Find the index of the element we want to delete

2. Swap this element with the last element

3. Down-heapify or up-heapify to restore the heap property. In a max-heap (min-heap), up-heapify is only required when the new key of element is greater (smaller) than the previous one because only the heap-property of the parent element might be violated. Assuming that the heap-property was valid between element and its children before the element swap, it can't be violated by a now larger (smaller) key value. When the new key is less (greater) than the previous one then only a down-heapify is required because the heap-property might only be violated in the child elements.

The decrease key operation replaces the value of a node with a given value with a lower value, and the increase key operation does the same but with a higher value. This involves finding the node with the given value, changing the value, and then down-heapifying or up-heapifying to restore the heap property.

Decrease key can be done as follows:

1. Find the index of the element we want to modify

2. Decrease the value of the node

3. Down-heapify (assuming a max heap) to restore the heap property

Increase key can be done as follows:

1. Find the index of the element we want to modify

2. Increase the value of the node

3. Up-heapify (assuming a max heap) to restore the heap property

Building a heap from an array of n input elements can be done by starting with an empty heap, then successively inserting each element. This approach, called Williams' method after the inventor of binary heaps, is easily seen to run in time: it performs n insertions at cost each.

However, Williams' method is suboptimal. A faster method (due to Floyd[8]) starts by arbitrarily putting the elements on a binary tree, respecting the shape property (the tree could be represented by an array, see below). Then starting from the lowest level and moving upwards, sift the root of each subtree downward as in the deletion algorithm until the heap property is restored. More specifically if all the subtrees starting at some height have already been "heapified" (the bottommost level corresponding to ), the trees at height can be heapified by sending their root down along the path of maximum valued children when building a max-heap, or minimum valued children when building a min-heap. This process takes operations (swaps) per node. In this method most of the heapification takes place in the lower levels. Since the height of the heap is , the number of nodes at height is . Therefore, the cost of heapifying all subtrees is:

This uses the fact that the given infinite series converges.

The exact value of the above (the worst-case number of comparisons during the heap construction) is known to be equal to:

,[9][b]

The average case is more complex to analyze, but it can be shown to asymptotically approach comparisons.

The Build-Max-Heap function that follows, converts an array A which stores a complete binary tree with n nodes to a max-heap by repeatedly using Max-Heapify (down-heapify for a max-heap) in a bottom-up manner. The array elements indexed by , , ..., n are all leaves for the tree (assuming that indices start at 1)-thus each is a one-element heap, and does not need to be down-heapified. Build-Max-Heap runs Max-Heapify on each of the remaining tree nodes.

Heaps are commonly implemented with an array. Any binary tree can be stored in an array, but because a binary heap is always a complete binary tree, it can be stored compactly. No space is required for pointers; instead, the parent and children of each node can be found by arithmetic on array indices. These properties make this heap implementation a simple example of an implicit data structure or Ahnentafel list. Details depend on the root position, which in turn may depend on constraints of a programming language used for implementation, or programmer preference. Specifically, sometimes the root is placed at index 1, in order to simplify arithmetic.

Let n be the number of elements in the heap and i be an arbitrary valid index of the array storing the heap. If the tree root is at index 0, with valid indices 0 through n − 1, then each element a at index i has

Alternatively, if the tree root is at index 1, with valid indices 1 through n, then each element a at index i has

This implementation is used in the heapsort algorithm, where it allows the space in the input array to be reused to store the heap (i.e. the algorithm is done in-place). The implementation is also useful for use as a Priority queue where use of a dynamic array allows insertion of an unbounded number of items.

The upheap/downheap operations can then be stated in terms of an array as follows: suppose that the heap property holds for the indices b, b+1, ..., e. The sift-down function extends the heap property to b−1, b, b+1, ..., e. Only index i = b−1 can violate the heap property. Let j be the index of the largest child of a[ i] (for a max-heap, or the smallest child for a min-heap) within the range b, ..., e. (If no such index exists because then the heap property holds for the newly extended range and nothing needs to be done.) By swapping the values a[ i] and a[ j] the heap property for position i is established. At this point, the only problem is that the heap property might not hold for index j. The sift-down function is applied tail-recursively to index j until the heap property is established for all elements.

The sift-down function is fast. In each step it only needs two comparisons and one swap. The index value where it is working doubles in each iteration, so that at most log ₂ e steps are required.

For big heaps and using virtual memory, storing elements in an array according to the above scheme is inefficient: (almost) every level is in a different page. B-heaps are binary heaps that keep subtrees in a single page, reducing the number of pages accessed by up to a factor of ten.

The operation of merging two binary heaps takes Θ( n) for equal-sized heaps. The best you can do is (in case of array implementation) simply concatenating the two heap arrays and build a heap of the result. A heap on n elements can be merged with a heap on k elements using O(log n log k) key comparisons, or, in case of a pointer-based implementation, in O(log n log k) time. An algorithm for splitting a heap on n elements into two heaps on k and n-k elements, respectively, based on a new view of heaps as an ordered collections of subheaps was presented in. The algorithm requires O(log n * log n) comparisons. The view also presents a new and conceptually simple algorithm for merging heaps. When merging is a common task, a different heap implementation is recommended, such as binomial heaps, which can be merged in O(log n).

Additionally, a binary heap can be implemented with a traditional binary tree data structure, but there is an issue with finding the adjacent element on the last level on the binary heap when adding an element. This element can be determined algorithmically or by adding extra data to the nodes, called "threading" the tree-instead of merely storing references to the children, we store the inorder successor of the node as well.

It is possible to modify the heap structure to allow extraction of both the smallest and largest element in time.[16] To do this, the rows alternate between min heap and max-heap. The algorithms are roughly the same, but, in each step, one must consider the alternating rows with alternating comparisons. The performance is roughly the same as a normal single direction heap. This idea can be generalized to a min-max-median heap.

In an array-based heap, the children and parent of a node can be located via simple arithmetic on the node's index. This section derives the relevant equations for heaps with their root at index 0, with additional notes on heaps with their root at index 1.

To avoid confusion, we'll define the level of a node as its distance from the root, such that the root itself occupies level 0.

For a general node located at index (beginning from 0), we will first derive the index of its right child, .

Let node be located in level , and note that any level contains exactly nodes. Furthermore, there are exactly nodes contained in the layers up to and including layer (think of binary arithmetic; 0111...111 = 1000...000 - 1). Because the root is stored at 0, the th node will be stored at index . Putting these observations together yields the following expression for the index of the last node in layer l.

Let there be nodes after node in layer L, such that

Each of these nodes must have exactly 2 children, so there must be nodes separating 's right child from the end of its layer ( ).

As required.

Noting that the left child of any node is always 1 place before its right child, we get .

If the root is located at index 1 instead of 0, the last node in each level is instead at index . Using this throughout yields and for heaps with their root at 1.

Every node is either the left or right child of its parent, so we know that either of the following is true.

Hence,

Now consider the expression .

If node is a left child, this gives the result immediately, however, it also gives the correct result if node is a right child. In this case, must be even, and hence must be odd.

Therefore, irrespective of whether a node is a left or right child, its parent can be found by the expression:

Since the ordering of siblings in a heap is not specified by the heap property, a single node's two children can be freely interchanged unless doing so violates the shape property (compare with treap). Note, however, that in the common array-based heap, simply swapping the children might also necessitate moving the children's sub-tree nodes to retain the heap property.

The binary heap is a special case of the d-ary heap in which d = 2.

Here are time complexities of various heap data structures. Function names assume a min-heap. For the meaning of " O( f)" and " Θ( f)" see Big O notation.

# data[].attributes.attachment_link.description
A binary heap is a heap data structure that takes the form of a binary tree. Binary heaps are a common way of implementing priority queues. The binary heap was introduced by J. W. J. Williams in 1964, as a data structure for heapsort.

# data[].attributes.headline
Tutorial en inglés de algoritmos de ordenamiento

# data[].attributes.subject
Tutorial en inglés de algoritmos de ordenamiento

# data[].attributes.attachment_link.readable_text
Bubble Sort is a simple algorithm which is used to sort a given set of n elements provided in form of an array with n number of elements. Bubble Sort compares all the element one by one and sort them based on their values.

If the given array has to be sorted in ascending order, then bubble sort will start by comparing the first element of the array with the second element, if the first element is greater than the second element, it will swap both the elements, and then move on to compare the second and the third element, and so on.

If we have total n elements, then we need to repeat this process for n-1 times.

It is known as bubble sort, because with every complete iteration the largest element in the given array, bubbles up towards the last place or the highest index, just like a water bubble rises up to the water surface.

Sorting takes place by stepping through all the elements one-by-one and comparing it with the adjacent element and swapping them if required.

Implementing Bubble Sort Algorithm

Following are the steps involved in bubble sort(for sorting a given array in ascending order):

1. Starting with the first element(index = 0), compare the current element with the next element of the array.

2. If the current element is greater than the next element of the array, swap them.

3. If the current element is less than the next element, move to the next element. Repeat Step 1.

Let's consider an array with values {5, 1, 6, 2, 4, 3}

Below, we have a pictorial representation of how bubble sort will sort the given array.

So as we can see in the representation above, after the first iteration, 6 is placed at the last index, which is the correct position for it.

Similarly after the second iteration, 5 will be at the second last index, and so on.

Time to write the code for bubble sort:

// below we have a simple C program for bubble sort #include <stdio.h> void bubbleSort(int arr[], int n) { int i, j, temp; for(i = 0; i < n; i++) { for(j = 0; j < n-i-1; j++) { if( arr[j] > arr[j+1]) { // swap the elements temp = arr[j]; arr[j] = arr[j+1]; arr[j+1] = temp; } } } // print the sorted array printf("Sorted Array: "); for(i = 0; i < n; i++) { printf("%d ", arr[i]); } } int main() { int arr[100], i, n, step, temp; // ask user for number of elements to be sorted printf("Enter the number of elements to be sorted: "); scanf("%d", &n); // input elements if the array for(i = 0; i < n; i++) { printf("Enter element no. %d: ", i+1); scanf("%d", &arr[i]); } // call the function bubbleSort bubbleSort(arr, n); return 0; }

Although the above logic will sort an unsorted array, still the above algorithm is not efficient because as per the above logic, the outer for loop will keep on executing for 6 iterations even if the array gets sorted after the second iteration.

So, we can clearly optimize our algorithm.

Optimized Bubble Sort Algorithm

To optimize our bubble sort algorithm, we can introduce a flag to monitor whether elements are getting swapped inside the inner for loop.

Hence, in the inner for loop, we check whether swapping of elements is taking place or not, everytime.

If for a particular iteration, no swapping took place, it means the array has been sorted and we can jump out of the for loop, instead of executing all the iterations.

Let's consider an array with values {11, 17, 18, 26, 23}

Below, we have a pictorial representation of how the optimized bubble sort will sort the given array.

As we can see, in the first iteration, swapping took place, hence we updated our flag value to 1, as a result, the execution enters the for loop again. But in the second iteration, no swapping will occur, hence the value of flag will remain 0, and execution will break out of loop.

// below we have a simple C program for bubble sort #include <stdio.h> void bubbleSort(int arr[], int n) { int i, j, temp, flag=0; for(i = 0; i < n; i++) { for(j = 0; j < n-i-1; j++) { // introducing a flag to monitor swapping if( arr[j] > arr[j+1]) { // swap the elements temp = arr[j]; arr[j] = arr[j+1]; arr[j+1] = temp; // if swapping happens update flag to 1 flag = 1; } } // if value of flag is zero after all the iterations of inner loop // then break out if(flag==0) { break; } } // print the sorted array printf("Sorted Array: "); for(i = 0; i < n; i++) { printf("%d ", arr[i]); } } int main() { int arr[100], i, n, step, temp; // ask user for number of elements to be sorted printf("Enter the number of elements to be sorted: "); scanf("%d", &n); // input elements if the array for(i = 0; i < n; i++) { printf("Enter element no. %d: ", i+1); scanf("%d", &arr[i]); } // call the function bubbleSort bubbleSort(arr, n); return 0; }

In the above code, in the function bubbleSort, if for a single complete cycle of j iteration(inner for loop), no swapping takes place, then flag will remain 0 and then we will break out of the for loops, because the array has already been sorted.

Complexity Analysis of Bubble Sort

In Bubble Sort, n-1 comparisons will be done in the 1st pass, n-2 in 2nd pass, n-3 in 3rd pass and so on. So the total number of comparisons will be,

(n-1) + (n-2) + (n-3) + ..... + 3 + 2 + 1 Sum = n(n-1)/2 i.e O(n ²)

Hence the time complexity of Bubble Sort is .

The main advantage of Bubble Sort is the simplicity of the algorithm.

The space complexity for Bubble Sort is O(1), because only a single additional memory space is required i.e. for temp variable.

Also, the best case time complexity will be O(n), it is when the list is already sorted.

Following are the Time and Space complexity for the Bubble Sort algorithm.

Worst Case Time Complexity [ Big-O ]: O(n²)

Best Case Time Complexity [Big-omega]: O(n)

Average Time Complexity [Big-theta]: O(n²)

Space Complexity: O(1)

# data[].attributes.attachment_link.description
Bubble Sort is a simple algorithm which is used to sort a given set of n elements provided in form of an array with n number of elements. Bubble Sort compares all the element one by one and sort them based on their values.

# data[].attributes.body
http://www.cse.iitm.ac.in/~rupesh/teaching/hpc/jun16/4-openmp.pdf

# data[].attributes.body
https://www.oscarblancarteblog.com/2014/08/22/estructura-de-datos-arboles/

# data[].attributes.body
https://www.bigocheatsheet.com/

# data[].attributes.body
https://en.wikipedia.org/wiki/Sorting_algorithm

# data[].attributes.body
https://jariasf.wordpress.com/2012/03/02/algoritmo-de-busqueda-depth-first-search-parte-1/

# data[].attributes.body
https://stackoverflow.com/questions/44430194/roulette-wheel-selection-with-positive-and-negative-fitness-values-for-minimizat

# data[].attributes.body
http://www.cs.bham.ac.uk/internal/courses/intro-nc/current/notes/14-examples-of-evolutionary-algorithms.pdf

# data[].attributes.body
http://www.cs.ucc.ie/~dgb/courses/tai/notes/handout12.pdf

# data[].attributes.body
http://www.ra.cs.uni-tuebingen.de/mitarb/streiche/publications/Introduction_to_Evolutionary_Algorithms.pdf

# data[].attributes.body
https://www.sqlite.org/rescode.html

# data[].attributes.body
https://docs.oracle.com/cd/E19205-01/820-7883/index.html

# data[].attributes.body
https://people.sc.fsu.edu/~jburkardt/c_src/openmp/openmp.html

# data[].attributes.body
http://www.sc.ehu.es/acwarfra/arpar/LP/LP.fitxategiak/HOJAOMP.cas.pdf

# data[].attributes.body
http://jakascorner.com/blog/2016/07/omp-barrier.html

# data[].attributes.body
https://www.tutorialspoint.com/sqlite/sqlite_c_cpp.htm

# data[].attributes.body
https://www.cs.usask.ca/~spiteri/CMPT851/notes/openMP.pdf

# data[].attributes.body
https://www.openmp.org/wp-content/uploads/OpenMP4.0.0.Examples.pdf

# data[].attributes.headline
difference between omp critical and omp single

# data[].attributes.subject
difference between omp critical and omp single

# data[].attributes.body
https://stackoverflow.com/questions/33441767/difference-between-omp-critical-and-omp-single

# data[].attributes.body
http://pages.tacc.utexas.edu/~eijkhout/istc/istc.html

# data[].attributes.body
http://www.sc-camp.org/2014/thusday/OpenMPSlides_tamu_sc.pdf

# data[].attributes.body
https://www.openmp.org/wp-content/uploads/cspec20.pdf

# data[].attributes.body
https://www.openmp.org/wp-content/uploads/omp-hands-on-SC08.pdf

# data[].attributes.body
https://bisqwit.iki.fi/story/howto/openmp/

# data[].attributes.body
https://youtu.be/6jFkNjhJ-Z4

# data[].attributes.body
https://youtu.be/oiARwi0fJOU

# data[].attributes.body
https://www.sqlite.org/howtocompile.html

# data[].attributes.body
https://www.openmp.org/

# data[].attributes.body
https://en.wikipedia.org/wiki/OpenMP

# data[].attributes.body
https://gcc.gnu.org/onlinedocs/libgomp/index.html#SEC_Contents

# data[].attributes.body
https://www.devdungeon.com/content/sqlite3-tutorial

# data[].attributes.body
https://codebeautify.org/xmlviewer

# data[].attributes.body
https://beginnersbook.com/2018/10/xml-tutorial-learn-xml/

# data[].attributes.body
https://en.wikipedia.org/wiki/List_of_file_systems

# data[].attributes.body
http://elm-chan.org/fsw/ff/00index_e.html

# data[].attributes.body
https://www.oracle.com/database/what-is-database.html

# data[].attributes.body
http://staff.ustc.edu.cn/~csli/graduate/algorithms/book6/chap19.htm

# data[].attributes.body
Árbol B animado

https://www.cs.usfca.edu/~galles/visualization/BTree.html

# data[].attributes.attachment_link.readable_text
Unlike linear data structures (Array, Linked List, Queues, Stacks, etc) which have only one logical way to traverse them, trees can be traversed in different ways.

A Tree Data Structure can be traversed in following ways:

1. Depth First Search or DFS

1. Inorder Traversal

2. Preorder Traversal

3. Postorder Traversal

2. Level Order Traversal or Breadth First Search or BFS

3. Boundary Traversal

4. Diagonal Traversal

Algorithm Inorder(tree)

1. Traverse the left subtree, i.e., call Inorder(left->subtree)

2. Visit the root.

3. Traverse the right subtree, i.e., call Inorder(right->subtree)

Uses of Inorder Traversal:

In the case of binary search trees (BST), Inorder traversal gives nodes in non-decreasing order. To get nodes of BST in non-increasing order, a variation of Inorder traversal where Inorder traversal is reversed can be used.

Code implementation of Inorder traversal.
Output
Inorder traversal of binary tree is
4 2 5 1 3

Time Complexity: O(N)
Auxiliary Space: If we don't consider the size of the stack for function calls then O(1) otherwise O(h) where h is the height of the tree.

Algorithm Preorder(tree)

1. Visit the root.

2. Traverse the left subtree, i.e., call Preorder(left->subtree)

3. Traverse the right subtree, i.e., call Preorder(right->subtree)

Uses of Preorder:

Preorder traversal is used to create a copy of the tree. Preorder traversal is also used to get prefix expressions on an expression tree.

Code implementation of Preorder traversal:
Output
Preorder traversal of binary tree is
1 2 4 5 3

Time Complexity: O(N)
Auxiliary Space: If we don't consider the size of the stack for function calls then O(1) otherwise O(h) where h is the height of the tree.

Algorithm Postorder(tree)

1. Traverse the left subtree, i.e., call Postorder(left->subtree)

2. Traverse the right subtree, i.e., call Postorder(right->subtree)

3. Visit the root

Uses of Postorder:

Postorder traversal is used to delete the tree. Please see the question for the deletion of a tree for details. Postorder traversal is also useful to get the postfix expression of an expression tree

Below is the implementation of the above traversal methods:
Output
Postorder traversal of binary tree is
4 5 2 3 1

Some other Tree Traversals Techniques:

Some of the other tree traversals are:

For each node, first, the node is visited and then it's child nodes are put in a FIFO queue. Then again the first node is popped out and then it's child nodes are put in a FIFO queue and repeat until queue becomes empty.

Example:

Input: Level Order Treversal:
1
2 3
4 5

The Boundary Traversal of a Tree includes:

1. left boundary (nodes on left excluding leaf nodes)

2. leaves (consist of only the leaf nodes)

3. right boundary (nodes on right excluding leaf nodes)

In the Diagonal Traversal of a Tree, all the nodes in a single diagonal will be printed one by one.

Input :

Diagonal Traversal of binary tree:

8 10 14

3 6 7 13

1 4

Last Updated : 23 Jun, 2023

# data[].attributes.attachment_link.title
Tree Traversal Techniques - Data Structure and Algorithm Tutorials - GeeksforGeeks

# data[].attributes.attachment_link.description
A Computer Science portal for geeks. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions.

# data[].attributes.body
Práctica oficial 8: Árboles I

# data[].attributes.body
http://www.differencebetween.net/technology/difference-between-binary-tree-and-binary-search-tree/

# data[].attributes.body
https://blog.codemagic.io/what-you-can-do-with-yaml/

# data[].attributes.attachment_link.readable_text
Analysis of Recursion in Programming

In this blog, we will analyze the recursive algorithm using the Recurrence Tree Method and Master theorem. We will also discuss the advantages and disadvantages of recursion.

For every recursive algorithm, we can write recurrence relation to analyse the time complexity of the algorithm.

Recurrence relation of recursive algorithms

A recurrence relation is an equation that defines a sequence where any term is defined in terms of its previous terms.

The recurrence relation for the time complexity of some problems are given below:

Fibonacci Number

T(N) = T(N-1) + T(N-2) Base Conditions: T(0) = 0 and T(1) = 1

Binary Search

T(N) = T(N/2) + C Base Condition: T(1) = 1

Merge Sort

T(N) = 2 T(N/2) + CN Base Condition: T(1) = 1

Recursive Algorithm: Finding min and max in an array

T(N) = 2 T(N/2) + 2 Base Condition: T(1) = 0 and T(2) = 1

Karastuba algorithm for fast multiplication

T(N) = 3 T(N/2) + CN Base Condition: T(0) = 1 and T(1) = 1

Quick Sort

T(N) = T(i) + T(N-i-1) + CN

The time taken by quick sort depends upon the distribution of the input array and partition strategy. T(i) and T(N-i-1) are two smaller subproblems after the partition where i is the number of elements that are smaller than the pivot. CN is the time complexity of the partition process where C is a constant. .

Worst Case: This is a case of the unbalanced partition where the partition process always picks the greatest or smallest element as a pivot(Think!).For the recurrence relation of the worst case scenario, we can put i = 0 in the above equation.

T(N) = T(0) + T(N-1) + CN which is equivalent to T(N) = T(N-1) + CN

Best Case: This is a case of the balanced partition where the partition process always picks the middle element as pivot. For the recurrence relation of the worst case scenario, put i = N/2 in the above equation.

T(N) = T(N/2) + T(N/2-1) + CN which is equivalent to T(N) = 2T(N/2) + CN

Average Case: For average case analysis, we need to consider all possible permutation of input and time taken by each permutation.

T(N) = (for i = 0 to N-1) ∑ ( T(i) + T(N-i-1) ) / N

Note: This looks mathematically complex but we can find several other intuitive ways to analyse the average case of quick sort.

Analyzing the Efficiency of Recursive Algorithms

Step 1: Identify the number of sub-problems and a parameter (or parameters) indicating an input's size of each sub-problem (function call with smaller input size)

Step 2: Add the time complexities of the sub-problems and the total number of basic operations performed at that stage of recursion.

Note : Check whether the number of times the basic operation is executed can vary on different inputs of the same size; if it can, the worst-case, average-case, and best-case efficiencies must be investigated separately.

Step3: Set up a recurrence relation, with a correct base condition, for the number of times the basic operation is executed.

Step4: Solve the recurrence or, at least, ascertain the order of growth of its solution. There are several ways to analyse the recurrence relation but we are discussing here two popular approaches of solving recurrences:

Method 1: Recursion Tree Method

A recurrence tree is a tree where each node represents the cost of a certain recursive subproblem. We take the sum of each value of nodes to find the total complexity of the algorithm.

Steps for solving a recurrence relation

1. Draw a recursion tree based on the given recurrence relation.

2. Determine the number of levels, cost at each level and cost of the last level.

3. Add the cost of all levels and simplify the expression.

Let us solve the given recurrence relation by Recurrence Tree Method

From the above recurrence relation, we can find that

1. The problem of size N is divided into two sub-problems of size N/2.

2. The cost of dividing a sub-problem and then combining its solution of size N is CN.

3. Each time, the problem will be divided into half, until the size of the problem becomes 1.

The recursion tree for the above relation will be

Cost at each level

The cost of dividing and combining the solution of size N/2^k is N/2^k and the number of nodes at each level is 2^k.

So, the cost at each level is (N/2^k)*(N^k) = N.

Number of levels

Size of problem at level-0 : N/2^0 Size of problem at level-1 : N/2^1 Size of problem at level-2 : N/2^2 Size of problem at level-3 : N/2^3 . . . . Size of problem at level-k : N/2^k

Now, we know that the size of the problem at the last level is 1.

Let us suppose the last level be k. Then

 N/2^k = 1 N = 2^k Taking log both sides log(N) = log(2^k) log(N) = k*log(2) k = log2(N)

The number of levels in the recursion tree is log2(N).

Cost of the last level

The cost at the last level where the size of the problem is 1 and the number of subproblems is N.

cost at last level = N*T(1) = O(N)

Now, let's add up the cost at each level

= { N + N + N + ..logN times } + O(N) = O(NlogN) + O(N) = O(NlogN)

The time complexity of the above recurrence relation is O(N logN).

Method 2: Master theorem

Master theorem states that for a recurrence relation of form

T(N) = aT(N/b) + f(N) where a >= 1 and b > 1 If f(N) = O(N^k) and k ≥ 0, then Case 1: T(N) = O(N^logb(a)), if k < logb(a). Case 2: T(N) = O((N^k)*logN), if k = logb(a). Case 3: T(N) = O(N^k), if k > logb(a)

Note: We will skip the mathematical proof, but if you are interested in knowing the maths behind this, it's a task for you to look at the proof of this theorem.

Let's analyse the time complexity using the master theorem.

Example 1

The above recurrence relation is of binary search. Comparing this with master theorem, we get a = 1, b = 2 and k = 0 because f(N) = C = C(N^0)

Here logb(a) = k, so we can apply case 2 of the master theorem. (Think!)

T(n) = (N⁰*log(N)) = O(logN).

Example 2

The above recurrence relation is of merge sort . Comparing this with master theorem,a = 2, b = 2 and f(N) = CN. Comparing left and right sides of f(N), we get k = 1.

logb(a) = log2(2) = 1 = K

So, we can apply the case 2 of the master theorem. (Think!)

=> T(N) = O(N¹*log(N)) = O(NlogN).

Example 3

The above recurrence relation is of finding min-max in an array through the divide and conquer approach. Comparing this with master theorem,a = 2, b = 2 and f(N) = 1. Comparing left and right sides of f(N), we get k = 0.

logb(a) = log2(2) = 1 > k

So, we can apply the case 1 of the master thorem (Think!)

T(N) = O(N^logb(a)) = O(N¹) = O(N)

Example 4

Comparing this with master theorem, a = 2, b = 2 and f(N) = CN²

comparing left and right sides of f(n), we get k = 2.

logb(a) = log2(2) = 1 < k

So, we can apply the case 3 of the master thorem. (Think!)

T(N) = O(N^k)= O(N²)

Pros and Cons of recursive programming

Pros

Recursion can reduce time complexity.

Recursion adds clarity and reduces the time needed to write and debug code.

Recursion can lead to more readable and efficient algorithm descriptions.

Recursion is a useful way of defining things that have a repeated similar structural form like tree traversal.

Cons

Recursion uses more memory like using run-time stack.

Recursion can be slow.

If recursion is too deep, then there is a danger of running out of space on the stack and ultimately program crashes.

Critical Concepts to explore in Recursion

Different Types of recursion and properties

Iteration vs Recursion comparison

Why Stack Overflow error occurs in recursion?

How to convert recursive code into iterative code and vice versa?

Suggested Problems to solve in Recursion

Happy coding! Enjoy Algorithms.

# data[].attributes.attachment_link.description
In this blog, we will analyze the recursive algorithm using the Recurrence Tree Method and Master theorem. We will also discuss the advantages and disadvantages of recursion. For every recursive algorithm, we can write recurrence relation to analyse the time complexity of the algorithm.

# data[].attributes.body
https://sqlite.com/whentouse.html

# data[].attributes.body
https://sqlite.com/about.html