UNIT Data Structure Operations
A data structure is a systematic way of organizing and storing data so that it can be accessed and modified efficiently. Regardless of the specific structure (array, linked list, stack, tree, graph, etc.), most of them support a common set of fundamental operations.
Core Operations
- Traversing — visiting every element of the data structure exactly once, typically to display or process the data (e.g., printing all nodes of a linked list).
- Searching — finding the location of a given element/key within the structure (e.g., sequential or binary search).
- Insertion — adding a new element into the structure, at the beginning, end, or a specific position.
- Deletion — removing an existing element from the structure while keeping the remaining elements consistent.
- Sorting — arranging elements in a defined order (ascending or descending), using algorithms such as insertion sort, shell sort, or quick sort.
- Merging — combining two data structures of the same type (e.g., two sorted lists) into a single structure while preserving order.
- Updating — modifying the value of an existing element without changing its position.
Why it matters: the choice of data structure directly affects the time and space complexity of these operations. For example, insertion at the head of a linked list is O(1), while insertion at the head of an array is O(n) because all subsequent elements must shift.
Classification
| Type | Examples | Characteristic |
|---|---|---|
| Linear | Array, Linked List, Stack, Queue | Elements arranged sequentially |
| Non-linear | Tree, Graph | Elements connected hierarchically/arbitrarily |
| Homogeneous | Array | All elements of the same type |
| Non-homogeneous | Structures/Records | Elements of different types |
UNIT Fundamentals of Analysis of Algorithms & Efficiency
Algorithm analysis is the process of determining the amount of resources (time and space) an algorithm needs to run, as a function of the size of its input. The goal is to predict performance and compare alternative algorithms independent of hardware or implementation language.
Why Analyze Efficiency?
- To predict how an algorithm's running time grows as input size increases.
- To compare two or more algorithms that solve the same problem.
- To identify performance bottlenecks before implementation.
Measures of Efficiency
- Time complexity — number of basic operations (comparisons, assignments) executed as a function of input size n.
- Space complexity — amount of memory used by the algorithm, including auxiliary space, as a function of n.
Approaches to Analysis
- Worst-case analysis — the maximum time taken over all inputs of size n. Most commonly used since it gives an upper bound guarantee.
- Best-case analysis — the minimum time taken; usually not very informative on its own.
- Average-case analysis — the expected time over all possible inputs, assuming a probability distribution; harder to compute but realistic.
Counting Basic Operations — Example
Consider a loop that adds n numbers:
sum = 0
for i = 1 to n:
sum = sum + a[i]
return sum
The loop body executes exactly n times, so the algorithm has time complexity O(n) — linear in the input size.
UNIT Asymptotic Notations
Asymptotic notation describes the limiting behaviour of a function (the algorithm's running time) as the input size grows toward infinity, ignoring constant factors and lower-order terms.
Big-O Notation — O(g(n))
Represents the upper bound (worst case) of an algorithm's growth rate. f(n) = O(g(n)) if there exist positive constants c and n₀ such that:
0 ≤ f(n) ≤ c · g(n) for all n ≥ n₀
Example: if f(n) = 3n² + 5n + 2, then f(n) = O(n²).
Big-Omega Notation — Ω(g(n))
Represents the lower bound (best case) of an algorithm's growth rate. f(n) = Ω(g(n)) if there exist positive constants c and n₀ such that:
0 ≤ c · g(n) ≤ f(n) for all n ≥ n₀
Big-Theta Notation — Θ(g(n))
Represents a tight bound — used when an algorithm's running time is bounded both above and below by the same function (up to constant factors):
c₁ · g(n) ≤ f(n) ≤ c₂ · g(n) for all n ≥ n₀
Little-o and Little-omega
- o(g(n)) — a strict upper bound; f(n) grows strictly slower than g(n).
- ω(g(n)) — a strict lower bound; f(n) grows strictly faster than g(n).
Quick rule: Big-O ≈ "at most", Big-Omega ≈ "at least", Big-Theta ≈ "exactly (order of)".
UNIT Basic Efficiency Classes
Algorithms are commonly grouped into efficiency classes based on their growth rate. The table below lists them from most to least efficient.
| Class | Name | Example Algorithm |
|---|---|---|
| O(1) | Constant | Array index access |
| O(log n) | Logarithmic | Binary search |
| O(n) | Linear | Sequential search |
| O(n log n) | Linearithmic | Merge sort, Quick sort (avg.) |
| O(n²) | Quadratic | Insertion sort, Bubble sort |
| O(n³) | Cubic | Matrix multiplication (naive) |
| O(2ⁿ) | Exponential | Subset generation, brute-force TSP |
| O(n!) | Factorial | Permutation generation |
Note: as n grows large, even a small difference in efficiency class (e.g., O(n) vs O(n²)) leads to dramatically different running times — this is why asymptotic classification matters more than constant factors.
SEARCH Sequential (Linear) Search
Sequential search checks each element of the list one by one, starting from the first, until the target element is found or the list is exhausted. It works on both sorted and unsorted lists.
function sequentialSearch(arr, target):
for i = 0 to length(arr) - 1:
if arr[i] == target:
return i
return -1
Best case occurs when the target is the first element; worst case occurs when the target is the last element or absent from the list.
SEARCH Binary Search
Binary search works only on a sorted list. It repeatedly divides the search interval in half: compare the target with the middle element, and discard the half of the list that cannot contain the target.
function binarySearch(arr, target):
low = 0
high = length(arr) - 1
while low <= high:
mid = (low + high) / 2
if arr[mid] == target:
return mid
else if arr[mid] < target:
low = mid + 1
else:
high = mid - 1
return -1
Precondition: the array must be sorted before applying binary search; otherwise the halving logic gives incorrect results.
SEARCH Interpolation Search
Interpolation search improves on binary search for uniformly distributed, sorted data. Instead of always probing the middle, it estimates the probable position of the target using a formula similar to linear interpolation — closer to how we might guess a name's location in a phone book based on its first letter.
pos = low + ((target - arr[low]) * (high - low)) /
(arr[high] - arr[low])
function interpolationSearch(arr, target):
low = 0
high = length(arr) - 1
while low <= high and target >= arr[low] and target <= arr[high]:
if low == high:
if arr[low] == target: return low
return -1
pos = low + ((target - arr[low]) * (high - low)) /
(arr[high] - arr[low])
if arr[pos] == target:
return pos
else if arr[pos] < target:
low = pos + 1
else:
high = pos - 1
return -1
Worst case (O(n)) occurs when the data is not uniformly distributed, causing repeated poor position estimates — e.g., highly skewed values.
SEARCH Comparison & Analysis of Searching Techniques
| Technique | Data Requirement | Best | Average | Worst |
|---|---|---|---|---|
| Sequential Search | None (sorted or unsorted) | O(1) | O(n) | O(n) |
| Binary Search | Sorted | O(1) | O(log n) | O(log n) |
| Interpolation Search | Sorted & uniformly distributed | O(1) | O(log log n) | O(n) |
Key Takeaways
- Sequential search is simplest and works on any list, but scales poorly for large n.
- Binary search needs sorted data but gives a reliable logarithmic guarantee.
- Interpolation search can outperform binary search on uniformly distributed sorted data, but degrades to linear time on skewed distributions.
- For very large datasets where O(1) lookup is required regardless of order, hashing (below) is generally preferred over all three.
HASHING Hash Table & Hash Functions
A hash table is a data structure that maps keys to values using a hash function, allowing near-constant-time insertion, deletion, and lookup on average.
Hash Function
A hash function h(key) converts a key into an index within the bounds of the underlying array (the hash table). A good hash function should:
- Be fast to compute.
- Distribute keys uniformly across the table to minimize clustering.
- Minimize collisions (two different keys mapping to the same index).
Common Hash Function Methods
- Division method: h(key) = key mod m, where m is the table size (commonly a prime number).
- Multiplication method: uses the fractional part of key × A (0 < A < 1) to compute the index.
- Mid-square method: squares the key and extracts the middle digits as the index.
- Folding method: splits the key into parts, sums them, and uses the result (mod table size) as the index.
Load factor (α): α = n / m, where n is the number of stored entries and m is the table size. A high load factor increases the likelihood of collisions.
HASHING Collision Resolution Techniques
A collision occurs when two distinct keys hash to the same table index. Collision resolution techniques fall into two broad categories: open hashing and closed hashing.
Open Hashing (Open Addressing is NOT this — see note below)
Also called separate chaining. Each table slot holds a pointer to a linked list (or chain) of all keys that hash to that slot. On collision, the new entry is simply appended to the chain at that index.
Index 0 → [keyA] → [keyD] → null
Index 1 → [keyB] → null
Index 2 → [keyC] → [keyE] → [keyF] → null
- Insertion: compute h(key), append node to the chain at that index.
- Search/Deletion: compute h(key), then traverse the chain at that index.
- Advantage: table never "fills up"; performance degrades gracefully.
- Disadvantage: requires extra memory for pointers; long chains degrade lookup to O(n) in the worst case.
Closed Hashing (Open Addressing)
All entries are stored within the table itself — no chaining. On collision, the algorithm probes for the next available slot according to a defined sequence.
Probing Methods
- Linear Probing: h(key, i) = (h(key) + i) mod m. Checks the next slot sequentially. Simple, but prone to primary clustering (long runs of occupied slots).
- Quadratic Probing: h(key, i) = (h(key) + c₁i + c₂i²) mod m. Reduces primary clustering by probing slots at increasing quadratic distances.
- Double Hashing: h(key, i) = (h₁(key) + i · h₂(key)) mod m. Uses a second hash function to determine the probe interval, minimizing clustering further.
function insert(table, key):
i = 0
index = h(key)
while table[index] is occupied:
i = i + 1
index = (h(key) + i) mod m // linear probing
table[index] = key
- Advantage: no extra pointer memory; better cache locality.
- Disadvantage: table can become full; deletions require special handling (e.g., "tombstone" markers) to avoid breaking probe sequences.
Open vs Closed Hashing — Quick Comparison
| Aspect | Open Hashing (Chaining) | Closed Hashing (Probing) |
|---|---|---|
| Storage | Linked lists outside the table | Within the table array |
| Table can overflow? | No (chains grow) | Yes, once full |
| Memory overhead | Extra pointers | None extra |
| Deletion | Straightforward | Needs tombstones/care |
| Clustering issue | None | Possible (esp. linear probing) |
SORTING Insertion Sort
Insertion sort builds the final sorted array one element at a time. It takes each element and inserts it into its correct position among the previously sorted elements — much like sorting playing cards in your hand.
function insertionSort(arr):
for i = 1 to length(arr) - 1:
key = arr[i]
j = i - 1
while j >= 0 and arr[j] > key:
arr[j + 1] = arr[j]
j = j - 1
arr[j + 1] = key
Best case (already sorted): O(n), since the inner while loop never executes. Stable: Yes. In-place: Yes. Efficient for small or nearly-sorted datasets.
SORTING Shell Sort
Shell sort is a generalization of insertion sort that allows the exchange of elements that are far apart. It starts by sorting elements far apart from each other (using a "gap" sequence), then progressively reduces the gap, with the final pass being a standard insertion sort (gap = 1).
function shellSort(arr):
n = length(arr)
gap = n / 2
while gap > 0:
for i = gap to n - 1:
temp = arr[i]
j = i
while j >= gap and arr[j - gap] > temp:
arr[j] = arr[j - gap]
j = j - gap
arr[j] = temp
gap = gap / 2
*Average case depends on the chosen gap sequence (e.g., Shell's original sequence, Knuth's sequence). Stable: No. In-place: Yes. Shell sort moves elements over larger distances early on, reducing the total number of shifts compared to plain insertion sort.
Gap sequence matters: the original Shell sequence (n/2, n/4, ..., 1) gives O(n²) worst case, while sequences like Knuth's (3ᵏ−1)/2 can improve performance to roughly O(n^1.5).
SORTING Quick Sort
Quick sort is a divide-and-conquer algorithm. It selects a pivot element, partitions the array so that elements smaller than the pivot come before it and elements larger come after it, then recursively sorts the two partitions.
function quickSort(arr, low, high):
if low < high:
pivotIndex = partition(arr, low, high)
quickSort(arr, low, pivotIndex - 1)
quickSort(arr, pivotIndex + 1, high)
function partition(arr, low, high):
pivot = arr[high]
i = low - 1
for j = low to high - 1:
if arr[j] <= pivot:
i = i + 1
swap(arr[i], arr[j])
swap(arr[i + 1], arr[high])
return i + 1
Stable: No. In-place: Yes (auxiliary stack space for recursion only). Worst case O(n²) occurs when the pivot chosen is consistently the smallest or largest element (e.g., an already-sorted array with last-element pivot selection), causing maximally unbalanced partitions.
Avoiding worst case: using randomized pivot selection or the median-of-three rule (median of first, middle, and last elements) significantly reduces the chance of hitting the O(n²) worst case in practice.